Add ports to the Sortix repository.

This change imports the ports collection from the former porttix and srctix
repositories and converts them to port(5) files with metadata pointing to
the upstream release tarballs with patches checked into this repository.
Ports are now developed and versioned along with the operating system and
are automatically built per the PACKAGES environment variable. The patches
are licensed under the same license as the relevant ports.

Tix has gained support for the new port(5) format. tix-port(8) is the new
high level ports build even point that handles downloading pstream releases
into the new mirror cache directory, applying the patches, building the port
with the lower-level tix-build(8), and finally installing the binary
package. The new tix-vars(8) program parses port(5) files and the new
tix-rmdiff(8) program produces input for tix-rmpatch(8).

The old doc/ directory is discontinued in favor of manual pages documenting
the new ports system.

The obsolete porttix-create(8) and srctix-create(8) programs are removed.
This commit is contained in:
Jonas 'Sortie' Termansen 2022-06-13 22:29:53 +02:00
parent 81978ae938
commit 9588b0d3db
220 changed files with 23392 additions and 2422 deletions

8
.gitignore vendored
View File

@ -5,10 +5,12 @@
/*.so
/*.a
builds
mirror
ports/*/*/
ports/*/*.version
release
repository
sysroot
sysroot.destdir
sysroot-overlay
sysroot-overlay-*
release
ports
repository

View File

@ -1,4 +1,4 @@
Copyright 2011-2016 Jonas 'Sortie' Termansen and contributors.
Copyright 2011-2022 Jonas 'Sortie' Termansen and contributors.
Permission to use, copy, modify, and distribute this software for any
purpose with or without fee is hereby granted, provided that the above
@ -26,7 +26,8 @@ software. There is no code under the GNU GPL or variants.
The Sortix ports collection contains third party code which is licensed
independently of the Sortix base systems. Such code may have licenses that
impose restrictions, such as requiring also providing the corresponding source
code when providing compiled program code to others.
code when providing compiled program code to others. Patches to ports are
licensed under the same license as the relevant port.
Official Sortix releases contain the binary packages from the ports collection
and thus may contain code whose licenses impose restrictions. Official Sortix

View File

@ -4,7 +4,6 @@ include build-aux/compiler.mak
include build-aux/version.mak
MODULES=\
doc \
libc \
libm \
dispd \
@ -43,11 +42,14 @@ ifndef SYSROOT_OVERLAY
endif
SORTIX_BUILDS_DIR?=builds
SORTIX_MIRROR_DIR?=mirror
SORTIX_PORTS_DIR?=ports
SORTIX_RELEASE_DIR?=release
SORTIX_REPOSITORY_DIR?=repository
SORTIX_ISO_COMPRESSION?=xz
SORTIX_PORTS_MIRROR?=https://pub.sortix.org/mirror
SORTIX_INCLUDE_SOURCE_GIT_REPO?=$(shell test -d .git && echo "file://`pwd`")
SORTIX_INCLUDE_SOURCE_GIT_REPO:=$(SORTIX_INCLUDE_SOURCE_GIT_REPO)
SORTIX_INCLUDE_SOURCE_GIT_ORIGIN?=
@ -225,15 +227,56 @@ endif
grep -E '^/src(/.*)?$$' | \
LC_ALL=C sort > "$(SYSROOT)/tix/manifest/src"
.PHONY: sysroot-ports
sysroot-ports: sysroot-fsh sysroot-base-headers sysroot-system sysroot-source
@SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
.PHONY: available-ports
available-ports:
@for port in $$(build-aux/list-packages.sh PACKAGES); do \
build-aux/upgrade-port.sh ports/$$port/$$port.port available; \
done
.PHONY: upgrade-ports
upgrade-ports:
@for port in $$(build-aux/list-packages.sh PACKAGES); do \
build-aux/upgrade-port.sh ports/$$port/$$port.port upgrade; \
done
.PHONY: mirror
mirror:
@SORTIX_MIRROR_DIR="$(SORTIX_MIRROR_DIR)" \
SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
SORTIX_REPOSITORY_DIR="$(SORTIX_REPOSITORY_DIR)" \
SORTIX_PORTS_MIRROR="$(SORTIX_PORTS_MIRROR)" \
SYSROOT="$(SYSROOT)" \
BUILD="$(BUILD)" \
HOST="$(HOST)" \
MAKE="$(MAKE)" \
MAKEFLAGS="$(MAKEFLAGS)" \
build-aux/build-ports.sh
build-aux/build-ports.sh download
.PHONY: extract-ports
extract-ports:
@SORTIX_MIRROR_DIR="$(SORTIX_MIRROR_DIR)" \
SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
SORTIX_REPOSITORY_DIR="$(SORTIX_REPOSITORY_DIR)" \
SORTIX_PORTS_MIRROR="$(SORTIX_PORTS_MIRROR)" \
SYSROOT="$(SYSROOT)" \
BUILD="$(BUILD)" \
HOST="$(HOST)" \
MAKE="$(MAKE)" \
MAKEFLAGS="$(MAKEFLAGS)" \
build-aux/build-ports.sh extract
.PHONY: sysroot-ports
sysroot-ports: sysroot-fsh sysroot-base-headers sysroot-system sysroot-source
@SORTIX_MIRROR_DIR="$(SORTIX_MIRROR_DIR)" \
SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
SORTIX_REPOSITORY_DIR="$(SORTIX_REPOSITORY_DIR)" \
SORTIX_PORTS_MIRROR="$(SORTIX_PORTS_MIRROR)" \
SYSROOT="$(SYSROOT)" \
BUILD="$(BUILD)" \
HOST="$(HOST)" \
MAKE="$(MAKE)" \
MAKEFLAGS="$(MAKEFLAGS)" \
build-aux/build-ports.sh build
.PHONY: sysroot
sysroot: sysroot-system sysroot-source sysroot-ports
@ -248,10 +291,13 @@ $(SORTIX_REPOSITORY_DIR)/$(HOST): $(SORTIX_REPOSITORY_DIR)
clean-core:
(for D in $(MODULES); do $(MAKE) clean -C $$D || exit $$?; done)
.PHONY: clean-mirror
clean-mirror:
rm -rf "$(SORTIX_MIRROR_DIR)"
.PHONY: clean-ports
clean-ports:
@SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
HOST="$(HOST)" \
MAKE="$(MAKE)" \
MAKEFLAGS="$(MAKEFLAGS)" \
build-aux/clean-ports.sh
@ -277,11 +323,18 @@ clean-sysroot:
.PHONY: clean
clean: clean-core clean-ports
.PHONY: distclean-ports
distclean-ports:
@SORTIX_PORTS_DIR="$(SORTIX_PORTS_DIR)" \
MAKE="$(MAKE)" \
MAKEFLAGS="$(MAKEFLAGS)" \
build-aux/clean-ports.sh distclean
.PHONY: mostlyclean
mostlyclean: clean-core clean-ports clean-builds clean-release clean-sysroot
mostlyclean: clean-core distclean-ports clean-builds clean-release clean-sysroot
.PHONY: distclean
distclean: clean-core clean-ports clean-builds clean-release clean-repository clean-sysroot
distclean: clean-core distclean-ports clean-builds clean-release clean-mirror clean-repository clean-sysroot
.PHONY: most-things
most-things: sysroot iso
@ -340,7 +393,13 @@ $(LIVE_INITRD): sysroot
mkdir -p $(LIVE_INITRD).d/home
mkdir -p $(LIVE_INITRD).d/root -m 700
cp -RT "$(SYSROOT)/etc/skel" $(LIVE_INITRD).d/root
cp doc/welcome $(LIVE_INITRD).d/root
(echo "You can view the documentation for new users by typing:" && \
echo && \
echo " man user-guide" && \
echo && \
echo "You can view the installation instructions by typing:" && \
echo && \
echo " man installation") > $(LIVE_INITRD).d/root/welcome
tix-collection $(LIVE_INITRD).d create --platform=$(HOST) --prefix= --generation=2
mkinitrd --format=sortix-initrd-2 $(LIVE_INITRD).d -o $(LIVE_INITRD)
rm -rf $(LIVE_INITRD).d
@ -491,7 +550,8 @@ $(SORTIX_RELEASE_DIR)/$(RELEASE)/repository/$(HOST):
.PHONY: release-repository
release-repository: sysroot $(SORTIX_RELEASE_DIR)/$(RELEASE)/repository/$(HOST)
for port in `LC_ALL=C ls "$(SYSROOT)/tix/tixinfo"`; do \
cp $(SORTIX_REPOSITORY_DIR)/$(HOST)/$$port.tix.tar.xz $(SORTIX_RELEASE_DIR)/$(RELEASE)/repository/$(HOST); \
cp $(SORTIX_REPOSITORY_DIR)/$(HOST)/$$port.tix.tar.xz $(SORTIX_RELEASE_DIR)/$(RELEASE)/repository/$(HOST) && \
cp $(SORTIX_REPOSITORY_DIR)/$(HOST)/$$port.version $(SORTIX_RELEASE_DIR)/$(RELEASE)/repository/$(HOST); \
done
.PHONY: release-scripts

View File

@ -1,4 +1,5 @@
#!/bin/sh -e
#!/bin/sh
set -e
make_dir_path_absolute() {
(cd "$1" && pwd)
@ -8,6 +9,18 @@ has_command() {
which "$1" > /dev/null
}
# Determine what's supposed to happen.
if [ $# = 0 ]; then
echo "$0: usage: $0 <operation>" >&2
exit 1
fi
OPERATION="$1"
case "$OPERATION" in
download|extract|build) ;;
*) echo "$0: error: Invalid operation: $OPERATION" >&2
exit 1
esac
# Detect if the environment isn't set up properly.
if [ -z "$HOST" ]; then
echo "$0: error: You need to set \$HOST" >&2
@ -18,25 +31,37 @@ elif [ -z "$SYSROOT" ]; then
elif [ -z "$SORTIX_PORTS_DIR" ]; then
echo "$0: error: You need to set \$SORTIX_PORTS_DIR" >&2
exit 1
elif [ -z "$SORTIX_MIRROR_DIR" ]; then
echo "$0: error: You need to set \$SORTIX_MIRROR_DIR" >&2
exit 1
elif [ -z "$SORTIX_REPOSITORY_DIR" ]; then
echo "$0: error: You need to set \$SORTIX_REPOSITORY_DIR" >&2
exit 1
elif ! [ -d "$SORTIX_PORTS_DIR" ]; then
echo "Warning: No ports directory found, third party software will not be built"
exit 0
elif ! has_command tix-collection ||
! has_command tix-build ||
! has_command tix-install; then
elif ! has_command tix-rmdiff; then
echo "$0: error: You need to have installed Tix locally to compile ports." >&2
exit 1
fi
# Create the mirror directory for downloaded archives.
mkdir -p "$SORTIX_MIRROR_DIR"
# Add the platform triplet to the binary repository path.
SORTIX_REPOSITORY_DIR="$SORTIX_REPOSITORY_DIR/$HOST"
mkdir -p "$SORTIX_REPOSITORY_DIR"
# Create the system root if absent.
if [ "$OPERATION" = build ]; then
mkdir -p "$SYSROOT"
fi
# Make paths absolute for later use.
SYSROOT=$(make_dir_path_absolute "$SYSROOT")
if [ "$OPERATION" = build ]; then
SYSROOT=$(make_dir_path_absolute "$SYSROOT")
fi
SORTIX_MIRROR_DIR=$(make_dir_path_absolute "$SORTIX_MIRROR_DIR")
SORTIX_PORTS_DIR=$(make_dir_path_absolute "$SORTIX_PORTS_DIR")
SORTIX_REPOSITORY_DIR=$(make_dir_path_absolute "$SORTIX_REPOSITORY_DIR")
@ -52,119 +77,71 @@ CXXFLAGS="$CXXFLAGS -Werror=format -Wno-error=format-contains-nul"
export CFLAGS
export CXXFLAGS
# Create the system root if absent.
mkdir -p "$SYSROOT"
# Create the binary package repository.
mkdir -p "$SORTIX_REPOSITORY_DIR"
# Initialize Tix package management in the system root if absent.
[ -e "$SYSROOT/tix/collection.conf" ] ||
tix-collection "$SYSROOT" create --platform=$HOST --prefix= --generation=2
# Detect all packages.
get_all_packages() {
for PACKAGE in $(ls "$SORTIX_PORTS_DIR"); do
! [ -f "$SORTIX_PORTS_DIR/$PACKAGE/tixbuildinfo" ] ||
echo $PACKAGE
done
}
# Detect which packages are available if not specified.
if [ -z "${PACKAGES+x}" ]; then
PACKAGES=$(get_all_packages | sort -R)
if [ "$OPERATION" = build ]; then
if [ ! -e "$SYSROOT/tix/collection.conf" ]; then
tix-collection "$SYSROOT" create --platform=$HOST --prefix= --generation=2
fi
fi
PACKAGES="$("$(dirname -- "$0")"/list-packages.sh PACKAGES)"
# Simply stop if there is no packages available.
if [ -z "$PACKAGES" ]; then
exit 0
fi
# Detect the build-time dependencies for a package.
get_package_dependencies_raw() {(
PACKAGE_DIR=$(echo $1 | grep -Eo '^[^\.]*')
! [ -f "$SORTIX_PORTS_DIR/$PACKAGE_DIR/tixbuildinfo" ] ||
grep -E "^(pkg\.build-libraries|pkg\.alias-of)=.*" "$SORTIX_PORTS_DIR/$PACKAGE_DIR/tixbuildinfo" | \
sed 's/^[^=]*=//'
)}
# Detect the build-time dependencies for a package with missing optional
# dependencies removed.
get_package_dependencies() {(
PRINTED_ANY=false
for DEPENDENCY in $(get_package_dependencies_raw $1); do
if [ "$DEPENDENCY" != "${DEPENDENCY%\?}" ]; then
DEPENDENCY="${DEPENDENCY%\?}"
FOUND=false
for PACKAGE in $PACKAGES; do
if [ "$PACKAGE" = "$DEPENDENCY" ]; then
FOUND=true
break
fi
done
if ! $FOUND; then
continue
fi
fi
if $PRINTED_ANY; then printf ' '; fi
printf "%s" "$DEPENDENCY"
PRINTED_ANY=true
done
if $PRINTED_ANY; then echo; fi
)}
# Decide the order the packages are built in according to their dependencies.
DEPENDENCY_MAKEFILE=$(mktemp)
(for PACKAGE in $PACKAGES; do
echo "$PACKAGE: $(get_package_dependencies $PACKAGE)"
echo " @echo $PACKAGE"
done;
printf ".PHONY:"
for PACKAGE in $PACKAGES; do
printf " $PACKAGE"
done;
echo) > "$DEPENDENCY_MAKEFILE"
if [ "$OPERATION" = build ]; then
PACKAGES="$(echo "$PACKAGES" | tr ' ' '\n' | sort -R)"
DEPENDENCY_MAKEFILE=$(mktemp)
(for PACKAGE in $PACKAGES; do
DEPENDENCIES="$(PACKAGES="$PACKAGES" \
"$(dirname -- "$0")"/list-packages.sh \
--dependencies $PACKAGE)"
echo "$PACKAGE: $(echo "$DEPENDENCIES" | tr '\n' ' ' | sed -E 's/ +$/\n/')"
echo " @echo $PACKAGE"
done;
printf ".PHONY:"
for PACKAGE in $PACKAGES; do
printf " $PACKAGE"
done;
echo) > "$DEPENDENCY_MAKEFILE"
BUILD_LIST=$(unset MAKE;
unset MFLAGS;
unset MAKEFLAGS;
make -Bs -f "$DEPENDENCY_MAKEFILE" $PACKAGES)
rm -f "$DEPENDENCY_MAKEFILE"
PACKAGES="$BUILD_LIST"
BUILD_LIST=$(unset MAKE;
unset MFLAGS;
unset MAKEFLAGS;
make -Bs -f "$DEPENDENCY_MAKEFILE" $PACKAGES)
rm -f "$DEPENDENCY_MAKEFILE"
PACKAGES="$BUILD_LIST"
fi
# TODO: This adds another decompression and compression to the build time, this
# should be done as a tix post installation step. Also this might miss
# programs in unusual locations, so need a thorough search and strip.
strip_tix() {
DIR=$(mktemp -d)
tar -C "$DIR" -xf "$1"
$HOST-strip -d "$DIR/data/bin/"* 2>/dev/null || true
$HOST-strip -d "$DIR/data/lib/"* 2>/dev/null || true
$HOST-strip -d "$DIR/data/libexec"* 2>/dev/null || true
$HOST-strip -d "$DIR/data/libexec/git-core/"* 2>/dev/null || true
$HOST-strip -d "$DIR/data/sbin/"* 2>/dev/null || true
(cd "$DIR" && tar --numeric-owner --owner=0 --group=0 -cJf port.tar.tix.xz tix data)
cp "$DIR/port.tar.tix.xz" "$1"
rm -rf "$DIR"
}
unset CACHE_PACKAGE
unset END
if [ "$OPERATION" = download ]; then
END=download
elif [ "$OPERATION" = extract ]; then
END=extract
else
CACHE_PACKAGE=--cache-package
fi
# Build all the packages (if needed) and otherwise install them.
# Build and install all the packages.
for PACKAGE in $PACKAGES; do
if ! [ -f "$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz" ]; then
SOURCE_PACKAGE=$(grep -E "^pkg.source-package=" "$SORTIX_PORTS_DIR/$PACKAGE/tixbuildinfo" | \
sed 's/^[^=]*=//')
tix-build \
--sysroot="$SYSROOT" \
--host=$HOST \
--prefix= \
--destination="$SORTIX_REPOSITORY_DIR" \
--generation=2 \
${SOURCE_PACKAGE:+--source-package "$SORTIX_PORTS_DIR/$SOURCE_PACKAGE"} \
"$SORTIX_PORTS_DIR/$PACKAGE"
strip_tix "$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz"
fi
tix-install \
SOURCE_PORT=$(tix-vars -d '' $SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.port \
SOURCE_PORT)
tix-port \
${BUILD:+--build="$BUILD"} \
$CACHE_PACKAGE \
--collection="$SYSROOT" \
--reinstall \
"$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz"
--destination="$SORTIX_REPOSITORY_DIR" \
${END:+--end="$END"} \
--generation=2 \
--host=$HOST \
${SORTIX_PORTS_MIRROR:+--mirror="$SORTIX_PORTS_MIRROR"} \
--mirror-directory="$SORTIX_MIRROR_DIR" \
--prefix= \
${SOURCE_PORT:+--source-port="$SORTIX_PORTS_DIR/$SOURCE_PORT/$SOURCE_PORT"} \
--sysroot="$SYSROOT" \
"$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE"
done

View File

@ -15,7 +15,7 @@ if [ -z "$SORTIX_PORTS_DIR" ]; then
elif ! [ -d "$SORTIX_PORTS_DIR" ] ||
[ "$(ls "$SORTIX_PORTS_DIR") | wc -l" = 0 ]; then
exit 0
elif ! has_command tix-build; then
elif ! has_command tix-vars; then
echo "$0: warning: Can't clean ports directory without Tix locally installed." >&2
exit 0
fi
@ -23,23 +23,41 @@ fi
# Make paths absolute for later use.
SORTIX_PORTS_DIR=$(make_dir_path_absolute "$SORTIX_PORTS_DIR")
# Detect all packages.
get_all_packages() {
for PACKAGE in $(ls "$SORTIX_PORTS_DIR"); do
! [ -f "$SORTIX_PORTS_DIR/$PACKAGE/tixbuildinfo" ] ||
echo $PACKAGE
done
}
# Clean all the packages.
for PACKAGE in $(get_all_packages); do
[ -f "$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz" ] ||
tix-build \
--sysroot="/" \
--host=$HOST \
--prefix= \
--destination="/" \
--start=clean \
--end=clean \
"$SORTIX_PORTS_DIR/$PACKAGE"
for PACKAGE in $("$(dirname -- "$0")"/list-packages.sh 'all!'); do
if [ "$1" = distclean ]; then
DEVELOPMENT=$(tix-vars -d false $SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.port \
DEVELOPMENT)
if [ "$DEVELOPMENT" = true ]; then
case "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.version" in
*.development)
echo "Port is in development: '$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE'"
continue
;;
esac
fi
if [ -e "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE" ]; then
echo "Removing '$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE'"
fi
rm -rf "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE"
rm -rf "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.upstream"
rm -f "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.version"
rm -f "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.version.new"
fi
if [ -e "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.version" -o \
-e "$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.version.new" ]; then
SOURCE_PORT=$(tix-vars -d '' $SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE.port \
SOURCE_PORT)
if [ -z "$SOURCE_PORT" ] ||
[ -e "$SORTIX_PORTS_DIR/$SOURCE_PORT/$SOURCE_PORT" ]; then
tix-build \
--sysroot="/" \
--prefix= \
--destination="/" \
--start=clean \
--end=clean \
${SOURCE_PORT:+--source-port "$SORTIX_PORTS_DIR/$SOURCE_PORT/$SOURCE_PORT"} \
"$SORTIX_PORTS_DIR/$PACKAGE/$PACKAGE"
fi
fi
done

View File

@ -115,7 +115,9 @@ live_initrd=$(maybe_compressed boot/live.initrd)
overlay_initrd=$(maybe_compressed boot/overlay.initrd)
src_initrd=$(maybe_compressed boot/src.initrd)
system_initrd=$(maybe_compressed boot/system.initrd)
ports=$(ls repository | sed 's/\.tix\.tar\.xz//')
ports=$(ls repository |
grep -E '\.tix\.tar\.xz$' |
sed -E 's/\.tix\.tar\.xz$//')
mkdir -p boot/grub
exec > boot/grub/grub.cfg

View File

@ -1,4 +1,5 @@
#!/bin/sh -e
#!/bin/sh
set -e
# Detect if the environment isn't set up properly.
if [ -z "$HOST" ]; then
@ -17,12 +18,11 @@ if ! [ -d "$SORTIX_REPOSITORY_DIR" ]; then
exit 0
fi
PACKAGES="$("$(dirname -- "$0")"/list-packages.sh PACKAGES)"
mkdir -p "$1"
if [ -z "${PACKAGES+x}" ]; then
cp -RT "$SORTIX_REPOSITORY_DIR" "$1"
else
for PACKAGE in $PACKAGES; do
cp "$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz" "$1"
done
fi
for PACKAGE in $PACKAGES; do
cp "$SORTIX_REPOSITORY_DIR/$PACKAGE.tix.tar.xz" "$1"
cp "$SORTIX_REPOSITORY_DIR/$PACKAGE.version" "$1"
done

108
build-aux/list-packages.sh Executable file
View File

@ -0,0 +1,108 @@
#!/bin/sh
set -e
if [ -z "$SORTIX_PORTS_DIR" ]; then
SORTIX_PORTS_DIR="$(dirname -- "$0")/../ports"
fi
# Load the ports sets.
. "$(dirname -- "$0")/ports.conf"
get_all_packages() {(
for package in $(ls "$SORTIX_PORTS_DIR"); do
if [ -f "$SORTIX_PORTS_DIR/$package/$package.port" ]; then
echo $package
fi
done
)}
get_package_dependencies_raw() {(
if [ -f "$SORTIX_PORTS_DIR/$1/$1.port" ]; then
tix-vars -d '' "$SORTIX_PORTS_DIR/$1/$1.port" BUILD_LIBRARIES
elif [ "$1" = "all" ]; then
get_all_packages
else
for set in $sets; do
if [ $set = $1 ]; then
echo $(eval echo \$set_$set)
fi
done
fi
)}
# TODO: This algorithm scales extremely poorly.
get_package_dependencies_recursive() {(
for dependency in $(get_package_dependencies_raw $1); do
want=false
if [ "$2" = "!!" ]; then
want=true
else
case "$dependency" in
*"?") ;;
*) want=true ;;
esac
fi
if $want; then
dependency=$(echo "$dependency" | tr -d '?')
# Optional dependencies might not exist yet.
if [ -f "$SORTIX_PORTS_DIR/$dependency/$dependency.port" ]; then
echo "$dependency"
get_package_dependencies_recursive "$dependency" "$2"
fi
fi
done
)}
list_dependencies() {(
package="$1"
for dependency in $(get_package_dependencies_raw "$package"); do
if [ "$dependency" != "${dependency%\?}" ]; then
dependency="${dependency%\?}"
for candidate in $PACKAGES; do
if [ "$candidate" = "$dependency" ]; then
echo "$dependency"
break
fi
done
else
echo "$dependency"
fi
done
)}
list_package() {(
package="$1"
# Fast path for listing all packages.
if [ "$package" = "all!" -o "$package" = "all!!" ]; then
get_all_packages
exit
fi
recursion=$(echo "$package" | grep -Eo '!*$')
package=$(echo "$package" | grep -Eo '^[^!]*')
# TODO: Better way of detecting if a port is a set.
if [ -f "$SORTIX_PORTS_DIR/$package/$package.port" ]; then
echo "$package"
fi
if [ -n "$recursion" ]; then
get_package_dependencies_recursive "$package" "$recursion"
fi
)}
if [ "$1" = "--dependencies" ]; then
shift
PACKAGES=$("$0" PACKAGES)
for package; do
list_dependencies "$package"
done | sort -u
exit
fi
for package; do
if [ "$package" = PACKAGES ]; then
for package in ${PACKAGES-all!}; do
list_package "$package"
done
else
list_package "$1"
fi
done | sort -u

103
build-aux/upgrade-port.sh Executable file
View File

@ -0,0 +1,103 @@
#!/bin/sh
set -e
NAME=$(tix-vars -d '' "$1" NAME)
COMPRESSION=$(tix-vars -d '' "$1" COMPRESSION)
DISTNAME_REGEX=$(tix-vars -d '' "$1" DISTNAME_REGEX)
RELEASE_SEARCH_PAGE=$(tix-vars -d '' "$1" RELEASE_SEARCH_PAGE)
RELEASE_SEARCH_REGEX=$(tix-vars -d '' "$1" RELEASE_SEARCH_REGEX)
UPSTREAM_ARCHIVE=$(tix-vars -d '' "$1" UPSTREAM_ARCHIVE)
UPSTREAM_SITE=$(tix-vars -d '' "$1" UPSTREAM_SITE)
VERSION=$(tix-vars -d '' "$1" VERSION)
VERSION_REGEX=$(tix-vars -d '' "$1" VERSION_REGEX)
if [ -z "$UPSTREAM_ARCHIVE" ]; then exit; fi
escape_regex() {
echo "$1" | sed -E 's,[\\+*?.{}<>],\\\0,g'
}
TAR_REGEX="(\.tar(\.(gz|bz2|xz)))"
if [ -z "$RELEASE_SEARCH_PAGE" ]; then
case "$UPSTREAM_SITE" in
https://github.com/*/releases/*)
RELEASE_SEARCH_PAGE=$(echo "$UPSTREAM_SITE" | grep -Eo '.*/releases/');;
*) RELEASE_SEARCH_PAGE="$UPSTREAM_SITE/";;
esac
fi
if [ -z "$VERSION_REGEX" ]; then
VERSION_REGEX="([0-9]+\.[0-9]+(\.[0-9]+)*)"
fi
if [ -z "$DISTNAME_REGEX" ]; then
DISTNAME_REGEX=$(echo "$UPSTREAM_ARCHIVE" |
sed -E 's,\+,\\+,g' |
sed -E "s,$VERSION_REGEX(\.tar.*),$(escape_regex "$VERSION_REGEX"),")
fi
if [ -z "$RELEASE_SEARCH_REGEX" ]; then
RELEASE_SEARCH_REGEX="\<$DISTNAME_REGEX$TAR_REGEX\>"
fi
upgrade_version() {
sed -E -e "s,^(VERSION_MAJOR)=.*,\\1=$(echo "$2" | sed -E 's,([0-9]+)\.([0-9]+)(\.([0-9]+))?.*,\1,')," \
-e "s,^(VERSION_MINOR)=.*,\\1=$(echo "$2" | sed -E 's,([0-9]+)\.([0-9]+)(\.([0-9]+))?.*,\2,')," \
-e "s,^(VERSION_PATCH)=.*,\\1=$(echo "$2" | sed -E 's,([0-9]+)\.([0-9]+)(\.([0-9]+))?.*,\4,')," \
-e "s/^(VERSION)=[^$]*$/\\1=$2/" \
-e "s/^(COMPRESSION)=.*/\1=$COMPRESSION/" \
"$1"
}
instantiate() {
upgrade_version "$1" "$2" | tix-vars - "$3"
}
LATEST=$(curl -Ls "$RELEASE_SEARCH_PAGE" | grep -Eo "$RELEASE_SEARCH_REGEX" | sort -Vu | tail -1)
case "$LATEST" in
*.tar) COMPRESSION=tar;;
*.tar.gz) COMPRESSION=tar.gz;;
*.tar.bz2) COMPRESSION=tar.bz2;;
*.tar.xz) COMPRESSION=tar.xz;;
esac
NEW_VERSION="$(echo "$LATEST" | sed -E "s,$RELEASE_SEARCH_REGEX,\1,")"
if [ -t 1 ]; then
RED='\033[91m'
GREEN='\033[92m'
RESET='\033[m'
else
RED=''
GREEN=''
RESET=''
fi
if [ -z "$NEW_VERSION" ]; then
printf "$RED%s$RESET\n" "$UPSTREAM_ARCHIVE failed to find available versions: $RELEASE_SEARCH_PAGE | grep -E '$RELEASE_SEARCH_REGEX'"
exit 1
fi
NEW_UPSTREAM_SITE=$(instantiate "$1" "$NEW_VERSION" "UPSTREAM_SITE")
NEW_UPSTREAM_ARCHIVE=$(instantiate "$1" "$NEW_VERSION" "UPSTREAM_ARCHIVE")
if ! wget -q "$NEW_UPSTREAM_SITE/$NEW_UPSTREAM_ARCHIVE" -O /dev/null; then
printf "$RED%s$RESET\n" "$NAME failed to download: $NEW_UPSTREAM_SITE/$NEW_UPSTREAM_ARCHIVE"
exit 1
fi
if [ "$VERSION" = "$NEW_VERSION" ]; then
echo "$NAME $VERSION is up to date"
exit
fi
if [ "$2" = upgrade ]; then
if [ "$(tix-vars -d false "$1" DEVELOPMENT)" = true ]; then
echo "$0: warning: Not upgrading $1 in DEVELOPMENT=true mode" >&2
exit
fi
(upgrade_version "$1" "$NEW_VERSION" && echo DEVELOPMENT=true) > "$1.new"
mv "$1.new" "$1"
fi
printf "$GREEN%s$RESET\n" "$NAME $VERSION -> $NEW_VERSION $NEW_UPSTREAM_SITE/$NEW_UPSTREAM_ARCHIVE"

View File

@ -93,7 +93,7 @@ verify_source() {
# TODO: Some of these patterns should also apply to build-aux, and maybe also
# partially to libm. Makefile and kblayout should also not have whitespace
# problems.
for MODULE in $(git ls-files | grep / | sed 's,/.*,,' | sort -u | grep -Ev '^(build-aux|doc|etc|libm|share)$'); do
for MODULE in $(git ls-files | grep / | sed 's,/.*,,' | sort -u | grep -Ev '^(build-aux|etc|libm|ports|share)$'); do
cd "$MODULE"
for FILE in $(git ls-files | grep -Ev '^include/' | grep -Ev '((^|/)(Makefile|\.gitignore|tixbuildinfo)|\.([0-9]|kblayout|f16|rgb))$'); do
verify_source "$FILE"

1
doc/.gitignore vendored
View File

@ -1 +0,0 @@
index.html

View File

@ -1,21 +0,0 @@
include ../build-aux/platform.mak
include ../build-aux/compiler.mak
include ../build-aux/version.mak
include ../build-aux/dirs.mak
DOCUMENTS:=\
cross-compilation-sins \
obsolete-stuff \
porting-guide \
posix-divergence \
welcome \
all: $(BINARIES)
.PHONY: all install clean
install: all
mkdir -p $(DESTDIR)$(DOCDIR)
cp -t $(DESTDIR)$(DOCDIR) $(DOCUMENTS)
clean:

View File

@ -1,268 +0,0 @@
Cross Compilation Sins
======================
Cross-compilation is the act of compiling a program for execution on another
machine that is different from the current machine. The other machine can have
another processor, another operating system, or just another directory structure
with different programs and libraries installed. The important part is that the
cross-compiled program is not meant to be executed on the build machine and
cannot be assumed to execute correctly on the build machine.
Cross-compilation works in concept simply by substituting the usual compiler
with a special cross-compiler targeting the host machine rather than the build
machine. The development headers and libraries would then already be installed
in the system root used by the cross-compiler. The program will then be built
normally and the result is a binary for the host machine.
Unfortunately, a lot of software attempt to be too clever for their own good and
rely on assumptions that are not valid in the face of cross-compilation. This is
a list of such sins that software packages occasionally make that prevent proper
cross-compilation. Almost all software ought to cross-compile cleanly.
Not supporting cross-compilation
--------------------------------
Programs should use a build system that allows cross-compiler, or be stupid and
just use the specified compiler without violating rules in this document. Build
systems should not hard-code a particular compiler, but use a CC or CXX
environmental variable that possibly default to reasonable values, or using the
full ./configure machinery for detecting the correct compiler.
Root is the empty prefix
------------------------
The build system must treat the empty prefix as distinct from the default prefix
(which is usually /usr/local). A prefix is a string of characters that is added
before the usual installation directories such as /bin and /lib. To install a
program into the /bin directory (in the root filesystem), you need add no
characters in front of /lib. Thus, the empty prefix is the root directory. The
root directory is the prefix of /, as that would install the program into //bin.
Indeed //bin is not only ugly, but it actually has a special alternate meaning
than the root directory on some operating systems.
Probing root filesystem
-----------------------
The actual root filesystem of the host machine is not available at build time.
That means that probing system files are out of the question and should be
recoded into a runtime check at program startup. For instance, checking special
directories such as /dev or /proc, or system configuration in /etc, even
manually locating headers in /usr/include and libraries in /usr/lib is not
allowed (when though they would be there in the system root, potentially).
Instead build systems should view things through the eyes of the cross-compiler
that may use exotic strategies to locate the desired resources (perhaps the
cross-compiler is an executable that wraps the real compiler and adds special
-I and --sysroot options, for instance). To check whether a header is available,
the build system should preprocess a file that #includes it. To check whether a
library is available, the build system should try to link an executable against
it.
This discussion even covers looking at the configured prefix and other
installation directories, even if prefixed with $DESTDIR, except for the purpose
of installing files there.
Executing cross-compiled programs
---------------------------------
It is not possible to run a cross-compiled program when cross-compiling. The
results vary, maybe the program works, maybe the program loads but does
something unexpected, maybe the program loads and enter an infinite loop, maybe
the program loads and crashes, or maybe the program doesn't even load. The
important part is that the build system should not even attempt to execute any
programs it compiles. Some packages use special configure-time tests to compile
test programs to check the behavior of the host machine, such as whether a
standard function works.
Such tests are fundamentally broken if they require executing the test program;
usually it is sufficient to just check if the program links. This means that
tests about the run-time behavior of programs are not possible on the build
machine. If possible, such tests could be delayed until program startup.
However, such tests are usually to detect broken systems. If testing is not
possible, then the build system should assume that the system is not broken. It
is often just a single operating system that has such a problem and it may be
fixed in later releases. It is acceptable to inconvenience users of broken
operating systems by asking them to pass an --using-broken-os configure option
or something similar, as long as it doesn't inconvenience honest users of good
operating systems.
$DESTDIR
--------
Programs must support the DESTDIR environmental variable at package installation
time. It is an additional temporary prefix that is added to all the installation
directory paths when installing files and directories. It is commonly used for
package management that wishes to stage programs at a temporary location rather
than installing into the final destination immediately. It is also used for
cross-compilation where installation into the local machine's root directory
would be disastrous, where you rather want to use the cross-compilation system
root as DESTDIR.
It is important to understand that the prefix set at configure time is the
location where the program will end up on the host machine, not the installation
directory on the build machine. The installation directory on the build machine
would be the prefix given at configure time prefixed with DESTDIR being the
system root.
If packages do not support DESTDIR, it is possible to work-around the issue by
adding the intended DESTDIR to the prefix given at configure time. This works
only as long as the program doesn't remember the prefix or store it in system
configuration files. The better solution is just to patch in $(DESTDIR) support.
Cross-compiled Build Tools
--------------------------
Programs often need special-purpose build tools to compile some aspects of them
or associated files or documentation. If such a program is local to the project
and not a stand-alone tool on its own right, then it is not uncommon for the
build system to build first the tool and then use the tool during the build of
the real program.
This presents a crucial problem for cross-compilation: Two compilers must be in
play. Otherwise, the build system may helpfully use the cross-compiler to
cross-compile the build tool and attempt to execute the cross-compiled build
tool (which is intended to run on the host machine, not the build machine). This
is a common problem that prevents otherwise cross-compilable programs from
being cross-compilable.
The solution is to detect and use two compilers: The compiler for the build
machine and the compiler for the host machine. This introduces some new
complexity, but autoconf generated ./configure scripts can deal rather easily
with it. There is a problem, though, if the build tool itself has dependencies
on external projects besides the standard library, as that would mean the build
system would need to detect and handle dependencies for both the build and host
machine. I'm not even sure the autoconf configure style --with-foo options are
fine-grained enough to support --with-build-foo and with-host-foo options if
they differ.
A better solution is perhaps to promote the custom build tool to a general
purpose or at least special purpose tool that is installed along with the
program, while allowing the special purpose tool to be built separately and
just that tool. This allows the user wishing to cross-compile to first build the
custom build tool locally on his configure, install it on the build machine, and
then have the tool in $PATH during the actual cross-compilation of the program.
This way the build system doesn't need to have two compilers in play, but at the
cost of essentially splitting the project into two subprojects: The real program
and the custom build tool. This option is preferable if the custom build tool
can be adapted so it is reusable by other projects.
You can also change the implementation language of the build tool to an
interpreted language, such as a shell script, python or anything suitable. It
would be prudent to ensure such interpreted languages can also be cross-compiled
cleanly.
Degrading functionality of program
----------------------------------
Programs should not be partially cross-compilable, with optional parts of the
program not available if cross-compiled. In the event that such optional parts
cannot be cross-compiled, it might be because they are violating rules in this
document or an external dependency does.
Degrading quality of program
----------------------------
Programs that are cross-compiled should be as similar as possible to the case
where they are not cross-compiled. However, it is acceptable if there is a
performance loss if the program needs to do run-time checking when a test is not
possible at compile time or other cases where the build system needs to make a
decision and insufficient data is available and both solutions would work
correctly.
Custom Configure Scripts
------------------------
Some perhaps ship with a custom hand-written ./configure script as opposed to a
script generated by tools such as autoconf. It doesn't matter in what language
a configure script is written (as long as it can be correctly executed) or
whether it is generated or hand-written. However, it is important that it
correctly implement the standard semantics customary with GNU autoconf generated
./configure scripts. In particular, for this discussion, it must support the
--build, --host and --target options, as well as all the standard installation
directory options (--prefix, --exec-prefix, ...). It is also important that it
correctly locate a cross-compiler through the --host option by using it as a
cross-tool prefix. For instance, --host=x86_64-sortix must correctly locate
the x86_64-sortix-gcc compiler.
Remembering the Compiler
------------------------
Unusually, some libraries remember the compiler executable and used compiler
options and store them in special foo-config files or even installed system
headers. This must never be done as the cross-compiler is a temporary tool. The
library is meant to be used on the host system, it would be odd if programs
depending on the library attempted to use a cross-compiler when building such
programs on the host machine. It also violates the principle that which compiler
is used is decided by the user, rather than secretly by the package.
Making cross-compilation needlessly hard
----------------------------------------
Some build systems require the user to answer particular questions about the
host machine, while is perfectly capable of automatically answering such
questions about the build machine. This occasionally takes the form of autoconf
generated ./configure scripts requiring the user to set autoconf cache values
that answer runtime tests. The obvious solution is to do runtime tests at
program startup instead or more careful tests that are possible at compile time.
Other problems include all sorts of miscellaneous situations where the user is
required to jump through hoops to cross-compile a program, when it could have
been much simpler if the build system had followed standard patterns. This is
often a symptom of projects where cross-compilation is considered unusual and
special-purpose, rather than than a natural state of things if you don't assume
particular runtime tests are possible at compile time.
pkg-config
----------
Libraries should install pkg-config files rather than libtool .la files or
foo-config scripts as neither of those approaches support cross-compilation or
system roots, while pkg-config is perfectly aware of such use cases.
Programs should never look for libtool .la files or use foo-config scripts for
the same reasons. It is too possible that the program ends up finding a tool for
the build machine instead, if the installed foo-config script wasn't in the
user's PATH. Fortunately, the invocation of foo-config scripts and pkg-config
are usually similar enough, so it is simple to adapt a build system to use the
pkg-config variant exclusively instead.
The user can build a special cross-pkg-config or wrap an existing pkg-config by
setting special environmental variables. There are some caveats if the program
builds custom build tools that needs dependencies detected through pkg-config.
In that case, the user may need to have a special pkg-config with a tool prefix
or pass a configure option setting the name of the build machine pkg-config
script.
libtool .la files
-----------------
Libraries managed with libtool often install special .la files into the
configured libdir. These files contain information on how to link against the
library and what compiler options to use. Unfortunately they don't support
cross-compilation and system roots. As such, too easily the compile process will
begin attempting to link against files relative to the root directory on the
build machine.
The recommendation is to kill such files on sight and to never generate them in
the first place, and certainly to never install them into the system root. It is
usually safe to delete such files, especially if the library installs pkg-config
files or if you install into well-known system directories.
foo-config
----------
Libraries occasionally install special foo-config scripts into the configured
bindir. These files are executable shell scripts that output the compiler
options that should be used to link against the library. Unfortunately they
don't support cross-compilation and system roots. As such, too easily the
compile process will begin attempting to link against files relative to the root
directory on the build machine.
The recommendation is to kill such files on sight and to never generate them in
the first place, and certainly to never install them into the system root. It is
usually safe to delete such files, especially if the library installs pkg-config
files or if you install into well-known system directories. Watch out for
programs linking against the library that wrongly locate foo-config scripts in
your $PATH (which potentially come from your distribution). Such programs needs
to be patched to use pkg-config instead.

View File

@ -1,342 +0,0 @@
Obsolete Stuff
==============
One of the primary design goals of Sortix is to be a modern Unix system with the
old cruft removed. This means that some features/functions/headers in the C API
has been removed, never implemented in the first place, or is scheduled for
future removal. This is likely inconvenient, as a lot of programs still rely on
these obsolete/broken features, which will have to be fixed. Additionally, it'll
further complicate porting programs to Sortix. However, it is our belief that
ultimately such removals will be beneficial and this is merely a transitional
period. In almost all cases, we have made available superior interfaces that can
be used instead.
It may cause problems for portable programs that these features cannot or should
not be used, as lesser operating systems may not implement the modern
replacements. Either fix such systems or add some feature detection magic.
This is a list of common features that may currently be implemented, but that
you shouldn't use and if you do, then you should fix your program before it
breaks when the feature is finally removed.
You might be tempted to use a preprocessor conditional for __sortix__ to detect
the absence of these obsolete features and the availability of their modern
replacements. Keep in mind that other systems may have the modern replacements
and also may have removed features, and that they may supply other extensions to
replace the features. If you need to be portable, you should use whatever means
to automatically detect the availability of features at compile time, rather
than hardcode cases for each platform, as your software is likely to be ported
to platforms that didn't exist when you wrote your code.
asctime, asctime_r
------------------
This function is fundamentally broken. It doesn't support locales, it adds some
useless newline character at the end, the date format doesn't comply with ISO
standards, asctime is not thread safe, and so on. Actually, the POSIX standard
supplies code that implements the function because the format simply cannot
change. The function was actually already deprecated back in 1989 when the
original C standard was released. The solution is today the same as back then,
simply use the much superior strftime function instead. asctime_r tries to make
the function thread safe, but it doesn't fix the other broken semantics. The
only use of this function is to participate in protocols that somehow manages to
use asctime formatted dates, but then you might as well just paste in the POSIX
example code and hard code it in your program.
Sortix currently implement these functions for compatibility reasons.
creat
-----
Use open() instead of this poorly named function. Additionally, open() has a
similarly poorly named flag O_CREAT that does what you need.
Sortix currently implement this function for compatibility reasons.
clock
-----
The clock() function suffers from overflow issues where it wraps around and the
caller has to handle that, meaning it's not suitable for measuring long
intervals. Converting a clock interval to seconds it also bothersome and
requires division by CLOCKS_PER_SEC.
You should use clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &ts) instead as it has no
overflow issues, provides nanosecond precision if available, and can be combined
with APIs such as Sortix's <timespec.h> for convenient computing.
Sortix currently implements this function for compatibility reasons.
ctime, ctime_r
--------------
These functions are defined in terms of asctime and asctime_r. Therefore they
will need to be removed as well.
Sortix currently implement these functions for compatibility reasons.
F_GETLK, F_SETLK, F_SETLKW
--------------------------
These fcntl commands implement POSIX advisory file locking. Unfortunately, this
standard interface is very poorly designed. In particular, if a process closes
a file descriptor, then all locks the process has for that file is unlocked,
even though there might not be a lock associated with that file descriptor in
the first place. This means that if the main program locks /foo/bar and runs
a library routine that also happen to open /foo/bar, then the advisory lock set
up by the main program is silently gone when when the library routine closes the
file and returns to the main program. Additionally, the locks are attached to
processes, rather than file descriptors. This complicates using them for threads
and passing file locks onto child processes.
Use the flock (not to be confused with lockf) call instead as it works at a file
descriptor level.
ftime
-----
Use clock_gettime instead.
gethostbyname, gethostbyaddr
----------------------------
Use the protocol agnostic functions such as getaddrinfo(). For instance, if you
want a HTTP connection to www.example.com, do you really care how the data get
to and from there? Most of the time you want a reliable transport protocol to a
named host, but don't care much about the low-level details. If you use these
modern networking interfaces, then your program can use without modification
IPv4, IPv6, or whatever new protocol is used in the future.
gethostid, sethostid
--------------------
These functions are built on the assumption that 32-bits are enough such that
each computer has an unique identity. It isn't. These functions are usually
implemented by using the IPv4 address, which already creates conflicts because
multiple systems can have the same LAN address. The functions are silly and any
use of them probably is silly too.
getpgrp
-------
POSIX and BSD disagree on the function prototype for getpgrp. Use getpgid
instead, as everyone agrees on that.
gets
----
Use fgets or getline instead. This function has been removed in the latest C
standard, but most implementations carry it anyways. Curiously it is hated so
much that some compatibility libraries such as gnulib actively use magic to add
deprecation warnings not to use it, but these won't compile because gets isn't
declared in any Sortix headers.
gettimeofday
------------
Use clock_gettime instead.
getwd
-----
Use get_current_dir_name() or getcwd() instead. Don't rely on the existence of
PATH_MAX, but allocate buffers as needed.
isascii
-------
This function is rather pointless. If we use a character encoding that wasn't
ascii compatible, then it doesn't make sense. If we use a sane character
encoding such as UTF-8, then you can simply check if the value is at most 127.
inet_addr
---------
This function only supports IPv4 and it supports a variety of weird ways to type
IPv4 addresses. Use inet_pton instead, which supports other address families as
well and only support standard address notations.
inet_ntoa
---------
This function only supports IPv4 and it isn't thread safe. Use inet_ntop instead
which is thread safe and supports other address families.
lockf
-----
This function implements POSIX advisory locks. It suffers from the same basic
design mistakes that the fnctl advisory lock commands (F_GETLK, F_SETLK,
F_SETLKW) do and should be avoided for the same reasons (see above).
Use the flock (not to be confused with lockf) call instead as it works at a file
descriptor level.
mktemp
------
mktemp(3) (not the mktemp(1) utility) creates a unique path name, but creates no
file, and thus offers no guarantee that is unique with respect to other threads
and system processes. The function is racy and dangerous.
Use mkstemp(3) (or for directories mkdtemp(3)) instead.
PATH_MAX
--------
There is no such limit in Sortix. The kernel might restrict the path lengths at
some point, but that'll just be to protect against misbehaving processes. You
can use pathconf() or fpathconf() to see if a particular path has a limit, but
otherwise you should just allocate strings as much as needed. There should be
functions in place so you can use paths of any length. If you really need a
limit as a hack to fix a broken program, you can do something like:
#if !defined(PATH_MAX) && defined(__sortix__)
#define PATH_MAX 32768
#endif
If there is ever going to be a path limit, it'll probably be either this value
or higher. Ideally, your programs ought to work with paths of any reasonable
length.
putenv
------
This is a poorly designed interface for manipulating the environment which
interacts quite badly with interfaces such as setenv and unsetenv. The major
problem is that putenv makes the input string itself part of the environment,
but setenv makes a copy of the input string part of the environment. This means
that unsetenv (as well as putenv and setenv when changing an existing variable)
has to somehow know whether the a given entry in environ was allocated by setenv
and whether to free it. This isn't helped by the fact that the environ symbol
is publicly accessible and callers of putenv can change the environment by
editing the string the caller inserted. This means that the implementations of
setenv and unsetenv must do a considerable amount of book-keeping behind the
scenes to figure out whether a string was allocated by setenv or face memory
leaks when environment variables are changed or unset. The solution to get rid
of all the needless complexity putenv forces upon the other functions is simply:
Don't provide putenv in the first place and fix any software that uses putenv to
just call setenv instead.
sdl-config
----------
This SDL utility program is basically broken for cross-compilation and seems to
be a poor counterpart to to pkg-config. If you insist on using such config
tools, use pkg-config instead as it causes fewer problems. If you really need a
sdl-config script, implement it using pkg-config:
pkg-config "$@" sdl
The Sortix build system actually injects such a sdl-config into the PATH to make
sure programs don't use the wrong SDL libraries when cross-compiling.
setpgrp
-------
POSIX and BSD disagree on the function prototype for setpgrp. Use setpgid
instead, as everyone agrees on that.
settimeofday
------------
Use clock_settime instead.
select
------
The fd_set system is poorly designed and the FD_SETSIZE is considerably smaller
than INT_MIN on most systems, which violates that the value of the file
descriptor shouldn't matter as long as it is between 0 and INT_MAX. It would be
better to use poll instead. There is also the problem that select uses struct
timeval instead of the superior struct timespec, though pselect solves that
particular problem.
Sortix currently provides this function for compatibility reasons.
sprintf
-------
The sprintf function is dangerous as it can be hard to predict the length of the
output string safely. A mistake can easily end in security vulnerabilities and
undefined behavior. Use the snprintf function instead as it knows the size of
the destination buffer and safely truncates in the error case. Such truncation
can be detected by the caller. Use the asprintf function or another approach
if determining the output length is hard.
Sortix currently provides this function for compatibility reasons.
strings.h
---------
There must have been some confusion back in the day since this header was
created, rather than the functions just added to string.h. In sane
implementations, you can just include string.h that also declares these
functions. The strings.h header exists in Sortix for source-code compatibility,
but don't be surprised if it just includes the regular string.h.
struct timeval
--------------
This microsecond precision data structure has been fully replaced by struct
timespec, which offers nanosecond precision. All kernel APIs use struct timespec
exclusively.
Sortix currently provides this structure for compatibility reasons.
sys/param.h
-----------
This is a BSD header that contains a bunch of BSD-specific stuff and other
miscellaneous junk. The GNU libc implementation contains some useless macros
that doesn't justify its existence. The header inclusion can often be deleted
without any problems, but older systems may require its inclusion.
sys/time.h
----------
You don't need this header and it'll be removed at some point. It is filled with
obsolete functions and macros. The only reason you might want it is to get the
declaration of struct timeval, but that data type has been replaced by struct
timespec.
Sortix currently provides this header for compatibility reasons.
sys/timeb.h
-----------
This is a header that contains the ftime function that has been replaced, this
header has been removed as well.
times
-----
This function is badly designed and the whole clock_t and sysconf(_SC_CLK_TCK)
business is insane. It doesn't help there is problem with potential overflowing
and the accuracy of the function varies between systems. You should avoid this
function in favor of clock_gettime and the Sortix extension clocks that provide
the same information as struct timespec. If you need the atomic semantics of
times, you can use the Sortix extension timens.
Sortix currently provides this function for compatibility reasons.
tmpnam
------
There is an inherently race condition prone and has thread safely issues with a
NULL argument. Use tmpfile() instead if you can do with a file whose name you do
not know. Unfortunately, Sortix has yet no satisfying temporary file creation
function that gives you a file and its name without having to deal with silly
template strings and other problems.
utime
-----
Use utimens instead, or perhaps the more portable utimensat.
Sortix currently provides this function for compatibility reasons.
utimes
------
Use utimens instead, or perhaps the more portable utimensat.

View File

@ -1,85 +0,0 @@
Portability Sins
================
Sortix makes use of third-party software. In many cases porting software and
integrating it into the Sortix ports is a fairly smooth ride. Unfortunately,
this is not always the case if the package is flawed. Such mistakes are usually
not intentional, but are simply accidents, rarely tested code cases or simply
the upstream being unaware of portability issues. We should identify such issues
and report them to the upstream projects.
This is a list of common problems that cause problems when integrating packages
into the Sortix ports build system. Almost all the issues listed here will tend
to get patched in Sortix as we want good ports, not fragile ports that we would
need all sorts of tricks to work around.
TODO: Potentially merge with cross-compilation-sins as a package-sins guide.
Cross-compilation issues
------------------------
See doc/cross-compilation-sins as well.
Don't be different for the sake of it
-------------------------------------
Pay respect to the traditions unless you are strictly better.
distclean is what goes into a release
-------------------------------------
When finishing it's useful to be able to reset the source tree to the state that
you got it in (i.e. no temporary build files left behind) so you can easily diff
it against the upstream tarball. Packages usually have a makefile target called
distclean that cleans the source directory for distribution, ideal for such
purposes. However, in some packages the distclean target doesn't actually do
that: It occasionally deletes files present in the upstream tarball while
leaving other files behind that wasn't in the upstream tarball. This is pretty
annoying. It makes the Sortix patches for a port harder to read as they are
filled with noise (generated XML files are fun), unless manual care is taken.
The ideal is that you can extract a tarball and run configure, make and finally
make distclean. The source directory should then be equal to what is in the
tarball. Otherwise the actual distribution wasn't actually distribution-clean.
Fun examples: Deleting the configure script upon distclean, deleting the license
files upon distclean, calling the target dist-clean instead of distclean.
DESTDIR comes from the environment
----------------------------------
It comes from the environment.
Use setenv instead of putenv
----------------------------
Use the system malloc
Print system types in a portable manner
---------------------------------------
Use the proper casts and large types or Sortix extensions.
64-bit and JIT
--------------
Don't assume that mmap() always returns below 4 GiB and that the distance
between any two memory mappings are always less than 2 GiB. There are no such
guarantees on 64-bit systems, yet many just-in-time virtual machines assumes
this is true and truncate pointers if it is not (leading to obscure crashes).
Don't use seemingly-unused bits in types
----------------------------------------
For instance, on x86-64 the address space is currently actually only 48-bit
and the most significant 16-bits must always equal the 47th bit. Some see this
as 16-bit perfectly usable bits for their own purposes. Don't do this, it's
crazy and you know it. This often comes up in more subtle cases such as telldir
that returns an opaque value that might have any bits set, but usually doesn't,
which looks like bits that can be re-purposed. That is, until the system changes
a bit and the package explodes.
Don't do crazy stuff
--------------------
Just don't.

View File

@ -1,480 +0,0 @@
Porting Guide
=============
This guide documents how to port software to the Sortix operating system and how
to release your port to the Sortix community. Before you get started, you should
get the system source code and the current core set of existing ports and be
comfortable with building the system (with ports) entirely from scratch.
Overview
--------
### `$SORTIX_TOPLEVEL` ###
This is the top-level source directory containing the system source code. This
is actually not a real environmental variable, but just something used in this
document to denote the top-level system source code directory. This is where you
have cloned the Sortix source repository.
### Source Tix ###
A source tix is simply a directory, whose name is the name of a package, and
which contains the source code for the package and a special `tixbuildinfo`
file, which contains instructions for building the source tix into a binary tix.
### `$SORTIX_PORTS_DIR` (default `$SORTIX_TOPLEVEL/ports`) ###
This directory is where the build system will search for source tixes that
will automatically be built along with the rest of the system. Any other files
and sub-directories will be ignored. You can use symbolic links to source tixes
if you wish. You integrate new packages into the build simply by simply putting
the source tix inside this directory as a sub-directory. It will automatically
be built along with the rest of the system during the next system build.
### `.srctix.tar.xz` (Archived Source Tix) ###
Since you cannot publish raw directories, published source tixes are put into
a compressed archive and given the extension `.srctix.tar.xz`. You can easily
install an archived source tix by extracting it using standard tools:
cd $SORTIX_PORTS_DIR &&
tar --extract --file libfoo.srctix.tar.xz
This will install the source tix into the source repository and it will
automatically be built the next time you compile the system and ports.
The archived source tix is simply a tarball that contains a single directory,
which is a source tix as described above.
### `.porttix.tar.xz` (Archived Port Tix) ###
Upstream releases of software cannot be expected to contain a `tixbuildinfo`
file and sometimes they need to be patched. When maintaining a port, it is often
useful to have a copy of the upstream release and the patches applied to it.
The `.porttix.tar.xz` archives contain a copy of the upstream release (as a
compressed archive) and all the patches applied to it. This can automatically be
converted into an archived source tix using the `srctix-create` program. Users
will normally not use this format unless they wish to inspect how a package was
ported or confirm that no malicious alternations were made to the upstream
release.
Can the package be ported?
--------------------------
It's a very good idea to examine the package closely before attempting to port
it, or you might end up wasting a lot of time on something that isn't even
possible in the first place. There's a number of red flags that can prevent the
successful porting of a package, at least not without enhancements to the tix
package management system or to Sortix itself.
The first thing to verify is whether we want such a package on Sortix. Not all
packages are good, some have bad licenses, some have securit