New features with AN-2021-09-01:

This is the first localization step for the schily source consolidation. Many
programs now (hopefully) call gettext() for all strings that need localization.

-	The next step will include dgettext() calls for the libraries and the
	missing programs

-	The following step will include the extracted strings

-	The last step will include German translations and install support
	for the resulting binary message object files.

----------> Please test and report compilation problems! <---------

***** NOTE: As mentioned since 2004, frontends to the tools should *****
*****		call all programs in the "C" locale		   *****
*****		by e.g. calling: LC_ALL=C cdrecord ....		   *****
*****		unless these frontends support localized strings   *****
*****		used by the cdrtools with NLS support.		   *****

		*** WARNING        ***
		*** Need new smake ***

	*** Due to the fact that schily-tools 2014-04-03 introduced to use new macro
	*** expansions and a related bug fix in smake, you need a newer smake
	*** to compile this source. If your smake is too old and aborts, ensure to
	*** use the recent smake by calling:

	cd ./psmake
	./MAKE-all
	cd ..
	psmake/smake
	psmake/smake install

	The new smake version mentioned above is smake-1.2.4
	The recent smake version is smake-1.5

	*** Due to the fact that schily-tools 2018-01-26 introduced
	*** optimizations for the Schily version of SunPro Make, you
	*** need at least the dmake version from 2018/01/11 with support
	*** for the "export" directive to compile with this makefile system.

For the beginning of the list of new features of the software in this tarball, 
please scroll down to "NEW FEATURES"

	WARNING: the new version of the isoinfo program makes use of the 
		*at() series of functions that have been introduced by Sun
		in August 2001 and added to POSIX.1-2008. For older platforms,
		libschily now includes emulations for these functions but
		these emulations have not yet been tested thoroughly.
		Please report problems!

	BUG WARNING: Please never report bugs only to Linux distributions as
		they usually do not forward these bug reports upstream and
		the Linux distributions typically do not let skilled people
		check the bugs. We did not hear about a FIFO problem in star
		for a long time. Then a problem on Linux occurred once
		every 6000-10000 tries but it did not happen on Solaris after
		even 10 million tries, so it was not known besides Linux and
		not reported to the project.

	BUG WARNING: *** GNU make *** starts too early with parallel
		execution (when reading Makefiles and evaluating rules for
		"include" statements already). Since GNU make does not
		support a concept for a correct ordering of such actions,
		you need to be prepared to see GNU make fail in parallel
		mode. If you try to compile a maiden unpacked schilytools
		tarball in parallel mode using GNU make, this will definitely
		fail as a result of the GNU make timestamp caching bug. See
		below for more information.

		If you are interested in reliable parallel execution,
		it is recommended to use the included "dmake" program with
		a command line like:

			dmake -j10 -r -f SMakefile

		from the top level directory. Note that if you are on Linux,
		you need the dmake version from schilytools 2021-06-07 or
		newer, since that version introduced a solution for a kernel
		caused performance problem with filesystems on Linux. Older
		dmake versions will not be faster in parallel mode on Linux.

		The "dmake" program included in the schilytools tarball is the
		current version of the "new" SunOS make program that has been
		introduced in January 1986 by Sun Microsystems. It also
		introduced new features (like the "include" directive) that
		3 years later have been copied by gmake in a partially buggy
		way. As gmake does not fix showstopper bugs, it cannot be
		supported. Current showstoppers are: 1) gmake executes
		"include" related rules in the inverse order, causing rules
		to fail if they depend on files created by an "earlier" action
		2) gmake caches an outdated state of the directory and aborts
		with a wrong complain about allegedly missing files that in
		fact exist already, because they just have been remade.

NEW FEATURES:

-	autoconf: The new autoconf test whether SIGSTKSZ is a constant did not
	work because the array definition was inside main() and thus did
	unexpectedly work with compilers that support VLA ant thus support
	arrays of a non-constant size.

	We moved the array definition to outside of main() to fix that problem.

	Thanks to Jan Engelhardt for reporting.

-	autoconf: Added new tests to check for /bin/sh, /usr/bin/sh and 
	whether /bin is a symlink. This helps to create software that can 
	use a hardwired path to the default shell.

-	autoconf: Added a new test to check for /bin/posix/sh
	This detects the POSIX shell on HP-UX.

-	makefiles.tar.bz2 updated to match the current autoconf state.

-	SunPro Make: Modified files (compared to the original version released
	by Sun from December 2006) now explicitly mention CDDL version 1.0 
	only. This is because Oracle did publish a CDDL-1.1 with some changes 
	to 1.0 already and we do not know what a future CDDL might contain. 
	So this is mainly a precautionary measure to prevent Oracle from 
	taking code from this enhanced SunPro Make and hide it as closed 
	source after adding the code to their copy of SunPro Make.

	Before, only files with larger changes have been modified to mention
	"CDDL version 1.0 only".

	This is a change that reverts the CDDL header back to what Sun did use
	in Summer 2005 with the first OpenSolaris release and to what has been
	negotiated with the OpenSolaris community. The removal of
	"version 1.0 only" was a change that happened in the OpenSolaris base
	sources after Sun has been aquired by Oracle. We do not know why Sun
	did use a similar change for SunPro Make already when publishing it's
	source in December 2006, but the same CDDL header appeared with SCCS
	that has been initially released as OpenSource at the same time as
	SunPro Make.

	Since we did change SCCS in a significant way since then already 
	(less than 20% of the current SCCS code is from Sun/AT&T), that change 
	was applied to SCCS much earlier. SunPro Make now however has more 
	than 10% modified or new code and enters the state of a larger 
	divergence to the Sun original. This initially mainly were changes 
	related to added fixes for bugs that have not been touched by Sun 
	within 10 years and now is driven by the new code that was added to
	match the upcoming POSIX issue 8 release and code to add features 
	originally introduced by smake.

-	SunPro Make: removed an unneeded #define RW_NO_OVERLOAD_WCHAR

-	SunPro Make: Defining "extern char sys_siglist[]" is now based on
	HAVE_SYS_SIGLIST_DEF

-	SunPro Make: Now partially using a new SHELL_PATH definition, that is
	based on the new autoconf results: HAVE_SYMLINK__BIN, HAVE__BIN_SH,
	HAVE__USR_BIN_SH

-	SunPro Make: We are now using HAVE_SIGSET to set up a fallback from
	sigset() to signal().

-	SunPro Make: A workaround for the linux getopt() has been removed since
	we always use our libgetopt that is a Solaris based and enhanced
	getopt().

-	SunPro Make: Removed a special HP-UX treatment for the POSIX shell
	in favor of the new autoconf test result HAVE__BIN_POSIX_SH

-	SunPro Make: libmakestate/ld_file.c now uses the schily include files
	in order to be usable on platforms line z/OS without e.g. sys/param.h.

	Thanks to a report from Matthew R. Wilson

-	SunPro Make: The man page now better describes pattern macro replacement.

-	SunPro Make: The man page now .NO_PARALLEL, .NOTPARALLEL and .PARALLEL
	in make.1s as well.

-	SunPro Make: A new mode named "SUN" for SUN_MAKE_COMPAT_MODE has 
	been introduced. This switches make into compatibility with the Sun
	source as distributed on December 19 2006. Before, a similar effect
	could be achieved with "SUN_MAKE_COMPAT_MODE= make ..."

-	smake: The man page now better describes pattern macro replacement.

-	mdigest: Fixed a typo in the output from mdigest -help

	Thanks to Robert Clausecker for reporting.

-	libfind:
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	bsh: 
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	calc: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	calltree: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	change: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	compare: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	copy: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	count:
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	cut: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	fifo: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	label: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	match: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	mdigest: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	mountcd: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	mt: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	p: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	paste: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	sdd: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	sformat: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	sgrow: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	star:
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	star_sym:
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	strar:
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	smake: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	psmake: added symlinks to schily/locale.h and schily/nlsdefs.h

-	tartest: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	termcap: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	translit: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	udiff: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	ved: Added support for gettext() based translations
	The string "Jrg Schilling" is now separate and in a gettext() call.

-	With the gettext() related changes from above, it is possible to make
	the text "Jrg Schilling" readable even for people that use UTF-8 as
	character encoding instead of the base character encoding ISO-8859-1.

	Given that UNICODE is an extension to ISO-8859-1 (the values 0..255
	from ISO-8859-1 and UNICODE are identical), software should be able
	to display ISO-8859-1 characters correctly when in a UNICODE locale.

	This e.g. works with the schilytools pager p(1) and is based on the
	fact that printable ISO-8859-1 characters cause an EILSEQ error with
	mbtowc() when in an UTF-8 based locale. Printing such a character 
	unmodified and then incrementing the "next character pointer" by one
	character is the right way to go, but that unfortunately does not 
	seem to be implemented by other software.

	When UNICODE was introduced in 1992, UNICODE people claimed that UNICODE
	will solve all existing character displaying problems... I sayd: UNICODE
	will introduce just new character displaying problems.

	Since the above algorithm is not in the rendering software, where
	it should be, the only way to deal with the problem is to set up
	a gettext() translation from "Jrg Schilling" to "Jrg Schilling"
	in ISO-8859-1 encoding and let gettext() use iconv() to recode the
	result to a different encoding in case that the locally used locale is
	using a different encoding. This is done by creating the file
	"SCHILY_utils.po" with the following content:

msgid ""
msgstr ""
"Project-Id-Version: Schilytools\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2021-08-21 10:40+0200\n"
"Last-Translator: Jrg Schilling <EMAIL@ADDRESS>\n"
"Language: all\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=ISO-8859-1\n"
"Content-Transfer-Encoding: 8bit\n"

#, c-format
msgid "Jrg Schilling"
msgstr "Jrg Schilling"

	Then format that file into a file "SCHILY_utils.mo" using msgfmt(1)
	and install it for all languages. 

	IMPORTANT: *This* file is written in ISO-8859-1 and you need to
	do the file handling for "SCHILY_utils.po" and the creation of
	"SCHILY_utils.mo" in a ISO-8859-1 based locale in order to get
	correctly working *.mo files.

	To learn where you need to install "SCHILY_utils.mo" at least,
	install the schilytools by calling "make install" and then run

		truss p -version

	and check which files are opened. Typical locations are:

	"/opt/schily/share/locale/de_DE.ISO8859-1/LC_MESSAGES/SCHILY_utils.mo"

	or 

	"/opt/schily/share/locale/de_DE/LC_MESSAGES/SCHILY_utils.mo"

	or something similar.

	******
	Distributions that intend to deliver this "SCHILY_utils.po" as 
	mentioned in the step-by-step instructions from above should 
	install it and link it to all supported locale variants.
	******

	Note that future versions of the schilytools will include support for
	creating and installing gettext() based translations and then distros
	will no longer need to create this trick.


SCCS THOUGHTS:

-	SCCS: The current idea for converting a historic SCCS project into
	a project oriented SCCS history bundle is the following:

	-	Create a user map file for "sccslog" by calling:

		mkdir $HOME/.sccs
		$EDITOR $HOME/.sccs/usermap

		Enter the UNIX login names followed by a TAB, followed
		by an E-mail notation. Use one line per user, e.g.

			joerg	J. Schilling <joerg@mail.com>

	-	Create a copy of the whole project to work on for this test.
		Do not do this conversion on the original project until
		sccs-6.0 is ready.

	-	chdir to the project home directory of the just created copy.

	-	Call "sccs init -i ." to make the project using an in-tree
		project oriented repository.

	-	Call:

		find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 -

		for the CSRG BSD project use:

		find * -path '*SCCS/s.*' | TZ=US/Pacific /opt/schily/ccs/bin/sccscvt -NSCCS/s. -k -ooo -V6 -

		to convert all history files into SCCSv6 history files. The
		TZ=US/Pacific is important for the UCB conversion since SCCSv6
		uses timezones but SCCSv4 does not and we need to have the 
		correct timezone entries in the SCCSv6 history files.

		For the complete "schilytools" project with 4200 SCCS history
		files in 55 Mbytes, this takes 12 seconds for the SCCS history
		from 1984 .. 2020, but note that most of the edits from the
		1980s are lost, so there are few entries from the time
		before 1989.

		An alternate example: the SCCS history from the BSD-4.4 project
		from December 1979 up to June 1995 is in 12600 SCCS history
		files that take up 125 MB.
		The conversion time to the SCCSv6 history file format is
		18 seconds.

	-	Call:

		find * -path '*SCCS/s.*' | /opt/schily/ccs/bin/sccslog -changeset -

		to populate the changeset file from the existing deltas.

		For the complete "schilytools" project with 19600 commits,
		this takes 8 minutes. The resulting file .sccs/SCCS/s.changeset
		has a size of approx. 7 MBytes.

		An alternate example: the SCCS history from the BSD-4.4 project
		from December 1979 up to June 1995 has approx. 47000 commits.
		The conversion time is approx. 40 minutes.
		The size of the resulting changeset file is approx. 14 MBytes.

	-	convert the in-tree repository into an off-tree repository.
		This final step is not yet needed and there is currently no
		code to do that automatically.

	-	If you like to check the resulting changeset file, there is
		currently only one way to look at it, by calling:

		sccs -O get -p -A -m .sccs/SCCS/s.changeset | more

		This prints an annotated version of the changeset file.
		The next task is to develop an enhancement to "sccs log"
		that prints the changeset in a way similar to what "hg log -v"
		prints.

	-	NOTE: Normal filesystems on Linux are slow, it is advised to
		make the conversions on tmpfs for performance reasons in case
		you are using Linux.

	Please however keep in mind that this is still experimental and there is
	absolutely no grant that a changelog created with current experimental
	software will work correctly with the final SCCS version. The procedure
	is just an example to check how it may look like.

	The final conversion method will be more automated... most likely
	by a command similar to "sccs import ..."

	IMPORTANT: This is not yet the time to finally convert a project into
	the project mode, because the project would be stuck in the current 
	state. What we need to continue work in that repository state in the
	project mode is at least a working "sccs commit". Be prepared to remove
	the changeset history file once "sccs commit" works and to re-create
	the changeset file for that time.

 

-	SCCS TODO:

	-	Activate "fsdiff" as a "bdiff" replacement in delta(1)
		to speed up delta(1) and to reduce the size of the SCCS
		history files.

	-	Implement something that outputs similar information from
		the changeset file as printed with "hg log -v".

		This would be the next key feature.

	-	verify whether sccs.c uses -NSCCS in the back end programs
		correctly, instead of converting g-file names from the command
		line into s.file names in the frontend in order to forward 
		s.file names to the backend programs. This is needed for an
		off-tree repository.

		The related unit tests are already passed.

	-	Add code to to sccs(1) to send a list of files to admin(1) and
		delta(1) with new or modified files in order to have all
		important code for a "sccs commit" in a single program that
		does not need to deal with ARG_MAX limitations.

	-	Add code to admin(1), delta(1), sccs-log(1) and get(1) to 
		maintain/understand the changeset file.

		This is mainly writing out the sccschangeset(4) entries to an
		intermediate store if a single file has been treated
		successfully. For sccs-log(1), see below.

	-	Finish the work to allow normal line based diffs in SCCS even
		for binary files. This are files that include nul bytes and
		this needs to completely avoid fputs() and this needs an 
		initialized member p_line_length in struct packet even for 
		all content that does not result from a previous getline() call.

	-	sccs -R tell (and probably other subcommands?) does not yet
		work in NewMode

	-	Add code to libcomobj to understand the changeset file.
		This is needed in order to e.g. know the file names and file
		specific SIDs/state that corresponds to a project global SID.

	-	Find/verify a complete transactional model that allows to repair
		complex changes to the set of files for a project that have
		been aborted in the middle. The current idea is to create the
		file $PROJECTHOME/.sccs/changeset with the deltas to the
		changeset during a complex update operation.

	-	Find a decision on how to deal with the admin flags that are
		currently implemented as global flags and thus do not depend on
		the SID (version) if the history file.

	-	Aborting a transaction via ^C currently requires a manual
		removal of the global lock file. Find a way to avoid this in
		case that a commit has been aborted while being prompted for
		a commit message (which is before any real action happened).

	-	Implement a fully automated method to convert a SCCSv4 based
		history with unrelated history files into a new SCCSv6 based
		project mode history with a populated changeset history file.

		This will most likely be done as a variant of the to be defined
		new command "sccs sccsimport" that imports a whole existing old
		SCCS project.

	-	Implement this "sccs sccsimport" based conversion in a way where
		sccs(1) holds the global changeset lock for the whole time
		of the conversion.




-	Bourne Shell Missing features for POSIX compliance:

	- Support for $'...' quoting (this is not needed for the current
					version of POSIX but for the next POSIX
					version that will be named SUSv8).
					The development of SUSv8 will start in
					late 2016.

	We are now expecting the Bourne Shell to be fully POSIX compliant.

-	Bourne Shell further TODO list:

	-	Finish loadable builtin support.

	-	POSIX does not allow us to implement ". -h", so we will
		add a "source" builtin to be able to implement "source -h"

-	The following builtins (that are available in bsh) are still missing in
	the Bourne Shell:

	err			echo with output going to stderr
	glob			echo with '\0' instead of ' ' between args
	env			a builtin version of /usr/bin/env

	The following bsh intrinsics are still missing in the Bourne Shell:

	-			the restricted bsh has restriction features that
				are missing in the Bourne shell.

	-	source -h	read file into history but do not execute

	and probably more features not yet identified to be bsh unique.



Author:

Joerg Schilling
D-13353 Berlin
Germany

Email: 	joerg@schily.net

Please mail bugs and suggestions to me.
