New features with AN-2020-08-12:

This is the first localization step for the schily source consolidation. Many
programs now (hopefully) call gettext() for all strings that need localization.

-	The next step will include dgettext() calls for the libraries and the
	missing programs

-	The following step will include the extracted strings

-	The last step will include German translations and install support
	for the resulting binary message object files.

----------> Please test and report compilation problems! <---------

***** NOTE: As mentioned since 2004, frontends to the tools should *****
*****		call all programs in the "C" locale		   *****
*****		by e.g. calling: LC_ALL=C cdrecord ....		   *****
*****		unless these frontends support localized strings   *****
*****		used by the cdrtools with NLS support.		   *****

		*** WARNING        ***
		*** Need new smake ***

	*** Due to the fact that schily-tools 2014-04-03 introduced to use new macro
	*** expansions and a related bug fix in smake, you need a newer smake
	*** to compile this source. If your smake is too old and aborts, ensure to
	*** use the recent smake by calling:

	cd ./psmake
	./MAKE-all
	cd ..
	psmake/smake
	psmake/smake install

	The new smake version mentioned above is smake-1.2.4
	The recent smake version is smake-1.3

	*** Due to the fact that schily-tools 2018-01-26 introduced
	*** optimizations for the Schily version of SunPro Make, you
	*** need at least the dmake version from 2018/01/11 with support
	*** for the "export" directive to compile with this makefile system.


	WARNING: the new version of the isoinfo program makes use of the 
		*at() series of functions that have been introduced by Sun
		in August 2001 and added to POSIX.1-2008. For older platforms,
		libschily now includes emulations for these functions but
		these emulations have not yet been tested thoroughly.
		Please report problems!

	BUG WARNING: Please never report bugs only to Linux distributions as
		they usually do not forward these bug reports upstream and as
		the Linux distributions typically do not let skilled people
		check the bugs. We did not hear about a FIFO problem in star
		for a long time. Then a problem on Linux occurred once
		every 6000-10000 tries but it did not happen on Solaris after
		even 10 million tries, so it was not known besides Linux.

	BUG WARNING: *** GNU make *** starts too early with parallel
		execution (when reading Makefiles and evaluating rules for
		"include" statements already). Since GNU make does not
		support a concept for a correct ordering of such actions,
		you need to be prepared to see gmake fail in parallel
		mode. If you are interested in reliable parallel execution,
		it is recommended to use the included "dmake" program with
		a command line like:

			dmake -j10 -f SMakefile

		from the top level directory. Note that if you are on Linux,
		you need a halfway recent kernel or the compile time will not
		go down because of the low POSIX semaphore performance in
		older Linux kernels.

		The "dmake" program included in the schilytools tarball is the
		current version of the "new" SunOS make program that has been
		introduced in January 1986 by Sun Microsystems. It also
		introduced new features like the "include" directive that
		3 years later have been copied by gmake in a partially buggy
		way. As gmake does not fix showstopper bugs, it cannot be
		supported. Current showstoppers are: 1) gmake executes
		"include" related rules in the inverse order, causing rules
		to fail if they depend on files created by an "earlier" action
		2) gmake caches an outdated state of the directory and aborts
		with a wrong complain about allegedly missing files that in
		fact exist already.


-	autoconf: added a new test for the existence of <sys/auxv.h>

-	autoconf: added new tests for Linux getauxval() and FreeBSD elf_aux_info()

-	libschily: getexecpath.c now uses getauxval() on Linux and elf_aux_info()
	on FreeBSD. This was needed since readlink("/proc/curproc/file") on
	FreeBSD returns random values for hardlinked files, making it impossible
	to use the result in order to find out which behavior variant of a 
	fat binary is requested.

-	hdump/od: The automatic POSIX/non-POSIX switch based on a hardlink and
	a path that contains /xpg4/ and another that does not now also
	works on Linux and FreeBSD. This is a result of using
	getauxval()/elf_aux_info(). Before, this switch did only work on
	Solaris.

-	isaexec: The program now also supports Linux and freeBSD, but we do not
	yet have a way to read the supported ISA list on the Linux/FreeBSD.

	Are there ideas to to this?

-	star: GNU cpio archives may contain longer path names than permitted by
	the AT&T documentation. This did cause problems with our automated
	recognition of swapped CPIO headers. We now use a more robust method.

-	star: GNU cpio archives may contain longer path names than permitted by
	the AT&T documentation. This could cause star to dump core because the
	cpio module did not yet call the routine to grow our new dynamic path
	store.

-	star: older GNU compilers seem to decide that a division:

		long long / long results in a long.

	We now cast the divisor to long long to definitely get the right
	result for a printf argument.

-	star: changed a name in a parameter list for lreadlink() to avoid
	a GCC shadow warning.

-	star: The variable use_fifo is also needed if the FIFO is not
	available (e.g. on DOS). The previous code did not compile on platforms
	that did not support to implement the FIFO. We moved
	extern BOOL use_fifo out of a #ifdef FIFO

	Thanks to a hint from Albert Wik.

-	star: The file README.otherbugs has been changed to match the
	state of today and a geocrawler based URL now includes an
	archive.org prefix as geocrawler no longer exists.

-	cal: cal now works on platforms without i18n support. We now check 
	whether MON_1 and ABMON_1 are defined and otherwise leave the 
	english month names.

	Thanks to a hint from Albert Wik.

-	mkisofs: The test if (s_entry->size > (((off_t)190)*(off_t)0x3FFFF800))
	in tree.c is now only run if the platform supports largefiles at all.

	Thanks to a hint from Albert Wik.

-	patch: A call to signal(SIGCHLD, ...) has been put into #ifdef SIGCHLD
	for DOS.

	Thanks to a hint from Albert Wik.

-	Bourne Shell: Several #ifdef SIG* have been added to make it compile
	on older UNIX versions.

	Thanks to a hint from Albert Wik.

-	smake: Added some code to allow better debugging of MAKEFLAGS related
	problems.

-	smake: we now use strdup() to remember the content of the MAKEFLAGS
	environment as the content of a pointer into the environment array
	did change unexpectedly on DJGPP.

	Thanks to a hint from Albert Wik.

-	SCCS: libmpw added a xpopen() to allow pipes to stdin/stdout/stderr.

-	SCCS: Fixed a bug in libcombj introduced with last release: the 
	lock update function did not #include <signal.h> and thus did get
	the SA_RESTART definition only on platforms that pollute the namespace
	line Linux.

-	SCCS: All man pages now have a recent (complete) list of SCCS commands
	in the 	SEE ALSO section. This adds the man pages that recently have
	been written.

-	SCCS: Unit tests: The file sccs/tests/Makefile has been modified to
	allow the directories /tmp/SCCSTEST and /tmp/sccstests to be tmpfs 
	mountpoints or symlinks that point to a tmpfs mount. Without that trick,
	the SCCS unit tests take 50 minutes on Linux because of the slow
	Linux file systems, while they take 3 minutes on Solaris (if run on
	UFS or ZFS). With tmpfs, Linux is in a similar time range as
	Solaris.

-	SCCS: sccslog again allows ovelapping delgets and now allows
	overlapping delget groups to be correctly detected as commit bundle.
	Historical SCCS based projects constantly do this...e.g. the CSRG
	BSD SCCS history from Kirk McKusick, so we need to allow it.

-	SCCS: The buffer size for setvbuf() may now be modified by a
	compile time make argument for testing purposes.

-	SCCS: diff now uses getexecpath() from libschily if getexecname()
	is not available. On Solaris, this does not require libschily.

-	SCCS: struct packet now has a new member no_chksum.

-	SCCS: getline() no longer computes checksums at all in case
	gpkt->no_chksum has been set. This is used to speed up SCCS in
	some cases.

-	SCCS: delta no longer computes the checksum in the first pass if
	called with -Xprepend. This reduces the run time by 7% in case
	of updating larger changeset files.

-	SCCS: "Reading" the SCCS history files now is implemented using mmap()
	if possible. This speeds up operation by approx. another 15%.

-	SCCS: the function getline() now supports mmap().

-	SCCS: the function putline() now uses fwrite() instead of fputs()
	when in copy mode. This is needed since fputs() needs a nul byte
	at the end of the string that cannot be granted when using mmap().

	With these changes, we are now close to allow normal line based
	diffs in SCCS even for binary files. We would need to completely
	avoid fputs() and this needs an initialized member p_line_length
	in struct packet.

-	SCCS: the function sclose() now also unmap()'s the file if it
	was mmap()ed via sinit().

-	SCCS: the function sinit() now also mmap()s the file in case that
	it is smaller than 1G and the flag SI_NOMAP was not specified.

-	SCCS: new functions sseek() and stell() for libcomobj, to support
	the use of mmap().

-	SCCS: delta no longer closes and reopens the history file in case
	that bdiff(1) reports a segmentation problem and needs another
	try with smaller segments. This change was needed in order to
	support mmap() on the input file. Delta now instead calls
	grewind() that knows about mmap().

-	SCCS: sccscvt now calls sseek() instead of fseek() and stell()
	instead of ftell() in order to support mmap().

-	SCCS: val now calls sseek() instead of fseek() and stell()
	instead of ftell() in order to support mmap().

-	SCCS: admin now also supports the -Xgpath=name option previously
	introduced to delta.

	Thanks to a hint from Albert Wik.

-	SCCS: val added some more comment.

-	SCCS: The programs admin and delta removed some org_*hash named
	variables because they have not been used.

-	SCCS: sccslog now uses setvbuf() for a larger I/O size

-	SCCS: sccslog now uses getdelim() to support unlimited line length
	in the delta section even though this is improbable in the delta 
	table section.

-	SCCS: sccslog now moved the code to update the changeset file
	(used for converting to a project oriented SCCS) into a new
	function commit_changeset().

-	SCCS: sccslog now supports a new option usermap=file to specify a
	project specific mapping file. This helps e.g. to convert the
	CSRG BSD archive into modern SCCS by specifying a CSRG BSD
	specific map file.

-	SCCS: sccslog now supports a new option maxdelta=timespec that
	allows to configure the timespan for previous "sccs delget" calls
	that shall be bundled into a virtual commit. The syntax is the same
	as with -mtime xxx in modern find(1) implementations.

-	SCCS: sccslog now prevents the same file to appear more than
	once in the same commit bundle, except when using the new option
	-multfile.

-	SCCS: "sccs help log" now includes usermap=file, maxdelta=timespec
	-multfile and -R

-	SCCS: sccslog renamed struct xx into struct delt

-	SCCS: sccslog now correctly passes the GMT offset to the delta(1)
	program. The previous version did use a double dash by mistake in 
	case that the GMT offset was negative.

-	SCCS: sccslog now marks removed deltas in file names with an "R"
	in case that it is used to just list entries.

-	SCCS: sccslog no longer includes removed deltas into the changeset
	when converting an old SCCS project into a project oriented 
	bundle that forms a changeset.

-	SCCS: sccslog now implements a larger cache for user name to email
	translations. Before, only the last hit has been remembered, but
	the CSRG project has over 100 users.

-	SCCS: sccslog now uses the correct combination of SCCS delta comment,
	programmer name and already seen flag to create bundles of
	"sccs delget" calls as project bundled commits. This results in
	a sccslog that now correctly deals with overlapping commits and
	correctly handles the SCCS history from the CSRG BSD UNIX (1979-1995).
	The final result is a changeset history with monotone increasing
	time stamp list.

	WARNING: the correctness has only be verified when the conversion
	from the CSRG BSD SCCSv4 history files into SCCSv6 history files was
	done using TZ=US/Pacific. This could either be a still existing bug
	or caused by CSRG commits that have been done while switching into 
	Wintertime in Europe.


-	SCCS: The current idea for converting a historic SCCS project into
	a project oriented SCCS history bundle is the following:

	-	Create a user map file for "sccslog" by calling:

		mkdir $HOME/.sccs
		$EDITOR $HOME/.sccs/usermap

		Enter the UNIX login names followed by a TAB, followed
		by an E-mail notation. Use one line per user, e.g.

			joerg	J. Schilling <joerg@mail.com>

	-	Create a copy of the whole project to work on for this test.
		Do not do this conversion on the original project until
		sccs-6.0 is ready.

	-	chdir to the project home directory of the just created copy.

	-	Call "sccs init -i ." to make the project using an in-tree
		project oriented repository.

	-	Call:

		find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 -

		for the CSRG BSD project use:

		find * -path '*SCCS/s.*' | TZ=US/Pacific /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 -

		to convert all history files into SCCSv6 history files. The
		TZ=US/Pacific is important for the UCB conversion since SCCSv6
		uses timezones but SCCSv4 does not and we need to have the 
		correct timezone entries in the SCCSv6 history files.

		For the complete "schilytools" project with 4200 SCCS history
		files in 55 Mbytes, this takes 12 seconds for the SCCS history
		from 1984 .. 2020, but note that most of the edits from the
		1980s are lost, so there are few entries from the time
		before 1989.

		An alternate example: the SCCS history from the BSD-4.4 project
		from December 1979 up to June 1995 is in 12600 SCCS history
		files that take up 125 MB.
		The conversion time to the SCCSv6 history file format is
		18 seconds.

	-	Call:

		find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccslog -changeset -

		to populate the changeset file from the existing deltas.

		For the complete "schilytools" project with 19600 commits,
		this takes 8 minutes. The resulting file .sccs/SCCS/s.changeset
		has a size of approx. 7 MBytes.

		An alternate example: the SCCS history from the BSD-4.4 project
		from December 1979 up to June 1995 has approx. 47000 commits.
		The conversion time is approx. 40 minutes.
		The size of the resulting changeset file is aprox. 14 MBytes.

	-	convert the in-tree repository into an off-tree repository.
		This final step is not yet needed and there is currently no
		code to do that automatically.

	-	If you like to check the resulting changeset file, there is
		currently only one way to look at it, by calling:

		sccs -O get -p -A -m .sccs/SCCS/s.changset | more

		This prints an anannotated version of the changeset file.
		The next task is to develop an enhancement to "sccs log"
		that prints the changeset in a way similar to what "hg log -v"
		prints.

	-	NOTE: Normal filesystems on Linux are slow, it is advised to
		make the conversions on tmpfs for permformance reasons in case
		you are using Linux.

	Please however keep in mind that this is still experimental and there is
	absolutely no grant that a changelog created with current experimental
	software will work correctly with the final SCCS version. The procedure
	is just an example to check how it may look like.

	The final conversion method will be more automated... most likely
	by a command similar to "sccs import ..."

	IMPORTANT: This is not yet the time to finally convert a project into
	the project mode, because the project would be stuck in the current 
	state. What we need to continue work in that repository state in the
	project mode is at least a working "sccs commit". Be prepared to remove
	the changeset history file once "sccs commit" works and to re-create
	the changeset file for that time.

 

-	SCCS TODO:

	-	Implement something that outputs similar information from
		the changeset file as printed with "hg log -v".

		This would be the next key feature.

	-	verify whether sccs.c uses -NSCCS in the back end programs
		correctly, instead of converting g-file names from the command
		line into s.file names in the frontend in order to forward 
		s.file names to the backend programs. This is neded for an
		off-tree repository.

		The related unit tests are already passed.

	-	Add code to to sccs(1) to send a list of files to admin(1) and
		delta(1) with new or modified files in order to have all
		important code for a "sccs commit" in a single program that
		does not need to deal with ARG_MAX limitations.

	-	Add code to admin(1), delta(1), sccs-log(1) and get(1) to 
		maintain/understand the changeset file.

		This is mainly writing out the sccschangeset(4) entries to an
		imtermediate store if a single file has been treated
		succsessfully. For sccs-log(1), see below.

	-	Finish the work to allow normal line based diffs in SCCS even
		for binary files. This are files that include nul bytes and
		this needs to completely avoid fputs() and this needs an 
		initialized member p_line_length in struct packet even for 
		all content that does not result from a previous getline() call.

	-	sccs -R tell (and probably other subcommands?) does not yet
		work in NewMode

	-	Add code to libcomobj to understand the changeset file.
		This is needed in order to e.g. know the file names and file
		specific SIDs/state that corresponds to a project global SID.

	-	Find/verify a complete transactional model that allows to repair
		complex changes to the set of files for a project that have
		been aborted in the middle. The current idea is to create the
		file $PROJECTHOME/.sccs/changeset with the deltas to the
		changeset during a complex update operation.

	-	Find a decision on how to deal with the admin flags that are
		currently implemented as global flags and thus do not depend on
		the SID (version) if the history file.

	-	Aborting a transaction via ^C currently requires a manual
		removal of the global lock file. Find a way to avoid this in
		case that a commit has been aborted while being prompted for
		a commit message (which is before any real action happened).

	-	Implement a fully automated method to convert a SCCSv4 based
		history with unrelated history files into a new SCCSv6 based
		project mode history with a populated changeset history file.

		This will most likely be done as a variant of the to be defined
		new command "sccs sccsimport" that imports a whole existing old
		SCCS project.

	-	Implement this "sccs sccsimport" based conversion in a way where
		sccs(1) holds the global changeset lock for the whole time
		of the conversion.




-	Bourne Shell Missing features for POSIX compliance:

	- Support for $'...' quoting (this is not needed for the current
					version of POSIX but for the next POSIX
					version that will be named SUSv8).
					The development of SUSv8 will start in
					late 2016.

	We are now expecting the Bourne Shell to be fully POSIX compliant.

-	Bourne Shell further TODO list:

	-	Finish loadable builtin support.

	-	POSIX does not allow us to implement ". -h", so we will
		add a "source" builtin to be able to implement "source -h"

-	The following builtins (that are available in bsh) are still missing in
	the Bourne Shell:

	err			echo with output going to stderr
	glob			echo with '\0' instead of ' ' between args
	env			a builtin version of /usr/bin/env

	The following bsh intrinsics are still missing in the Bourne Shell:

	-			the restricted bsh has restriction features that
				are missing in the Bourne shell.

	-	source -h	read file into history but do not execute

	and probably more features not yet identified to be bsh unique.



Author:

Joerg Schilling
D-13353 Berlin
Germany

Email: 	joerg@schily.net, joerg.schilling@fokus.fraunhofer.de

Please mail bugs and suggestions to me.
