New features with AN-2019-01-22:

This is the first localization step for the schily source consolidation. Many
programs now (hopefully) call gettext() for all strings that need localization.

-	The next step will include dgettext() calls for the libraries and the
	missing programs

-	The following step will include the extracted strings

-	The last step will include German translations and install support
	for the resulting binary message object files.

----------> Please test and report compilation problems! <---------

***** NOTE: As mentioned since 2004, frontends to the tools should *****
*****		call all programs in the "C" locale		   *****
*****		by e.g. calling: LC_ALL=C cdrecord ....		   *****
*****		unless these frontends support localized strings   *****
*****		used by the cdrtools with NLS support.		   *****

		*** WARNING        ***
		*** Need new smake ***

	*** Due to the fact that schily-tools 2014-04-03 introduced to use new macro
	*** expansions and a related bug fix in smake, you need a new smake
	*** to compile this source. If your smake is too old and aborts, ensure this
	*** by calling:

	cd ./psmake
	./MAKE-all
	cd ..
	psmake/smake
	psmake/smake install

	The new smake version mentioned above is smake-1.2.4
	The recent smake version is smake-1.3

	*** Due to the fact that schily-tools 2014-01-26 introduced
	*** optimizations for the Schily version of SunPro Make, you
	*** need at least the dmake version from 2018/01/11 with support
	*** for the "export" directive to compile with this makefile system.


	WARNING: the new version of the isoinfo program makes use of the 
		*at() series of functions that have been introduced by Sun
		in August 2001 and added to POSIX.1-2008. For older platforms,
		libschily now includes emulations for these functions but
		these emulations have not yet been tested thoroughly.
		Please report problems!

-	Schily.Copyright: added new Copyright dates

-	include/schily/values.h: new include file.

-	include/schily/dbgmalloc.h: added more prototypes for libdbgmalloc.

-	libdbgmalloc & bsh: alloc.c now also prints the caller information
	in the function aprintlist()

-	libstrar::unicode.c: Code has been aligned again with the same module
	from star.

-	libstrar & star unicode.c: iconv() may return > 0 if there are
	characters that could not be converted into an identical meaning.
	We therefore now check for ret != 0 instead of ret == -1.

	Thanks to Christian Weisgerber for reporting

-	libschily::match.c: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	libschily::fstream.c: Added a /* NOTREACHED */ comment to make coverity
	quiet.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	libschily::fnmatch.c: added a forgotten fix (from last January) for 
	a problem with missing brackets with if/then/else related to
	[:upper:] and [:lower:]

-	libschily::fnmatch.c:  added a forgotten _partial_ enhancement from
	last January for [= ... =] and [. ... .] support.

	Warning: this is not yet fully functional since the POSIX standard
	does not include an interface to get this functionality from the
	i18n part of libc.

-	libfind::find.c: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	libshedit & bsh: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	calltree: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	calltree: made the lexer more robust against EOF in the middle of
	a token.

	Thanks to Heiko Eifeldt for reporting.

-	cdda2wav: replaced some strcat() calls by strlcat() calls.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	cdrecord: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	cdrecord: Copyright bumped to 2019

-	cpp: cpypre.c the fallback file for platforms without "yacc" has been
	modified to include schily/values.h instead of values.h. This is needed
	for platforms without yacc and without values.h (e.g. Cygwin)

	Thanks to Heiko Eifeldt for reporting.

-	cdda2wav & ${SRCROOT}/Mcsw & ${SRCROOT}/Mocsw is now using
		"$(MAKE)" instead of $(MAKE) to permit spaces in the path.

	Thanks to Heiko Eifeldt for reporting.

-	mkisofs: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	mkisofs: the Message "'iconv -l' lists more available names.\n" is now
	printed even when there is no locale support in the mkisofs specific
	directory: $INS_BASE/share/lib/siconv as on platforms with iconv(1),
	the iconv(1) specific translations are still possible.

	Thanks to Jan Engelhardt for reporting.

-	p: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	printf: Added a missing /* FALLTHROUGH */ comment.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: added support for auto detection of "zstd" compressed archives.

-	star: added a new option -zstd to support compression and uncompression
	using the program "zstd".

-	star: Recently, star did hang in the FIFO code on Solaris. This did
	not happen on Solaris over 20 years before...

	On Linux - on fast multi CPU machines - the probability that a
	child process from fork() starts up before the parent is 1000x higher
	than on Solaris, where 10 million tries were needed to reproduce the
	same problem.

	As a result, the FIFO in star on Linux could in rare cases (1 of 
	~ 10000 tries) even finish the 1st read() from the input file before
	the "tar"-process starts with e.g. command lines like "star -tv" or 
	"star -x". Since star introduced auto-byte-order detection and handling
	in 1985, star needs a special start up sequence to do that.

	Star introduced the FIFO in the late 1980s and the machines from that
	time did always restart the parent before the fork()ed child starts.
	The new OS behavior thus caused a situation that was not forseeable
	when the FIFO has been designed. This new OS behavior caused a
	deadlock in aprox. 1 of 10000 star calls on Linux and 1 of 10000000
	star calls on Solaris.

	Star now waits when entering the FIFO fill-process until the 
	FIFO get-process did start up before trying to wake up a waiting
	get process.

-	star: On Linux, in 1 of 1.5 million tries, star did die from SIGPIPE.
	Note that this did never happen on Solaris.

	Star now ignores SIGPIPE and it seems that this fixed the problem
	since it did not happen again after that change with even 100 million
	tries.

-	star: The debug printing for the FIFO has been enhanced to print more
	information from the FIFO control structure to make it easier to debug
	problems like the ones mentioned above.

-	star: There seems to be a problem in pipe handling in the Linux kernel.
	It seems that in rare cases, the read(2) on a pipe returns 0 even though
	the write side did write(2) one byte to the pipe just before calling
	exit(). Unfortunately, this problem is hard to debug as it happens only
	once every ~30 million tries. Our workaround is to behave as if the
	expected byte could be read and star currently prints something like:

		star: Erfolg. Sync pipe read error pid 8141 ret 0
		star: Erfolg. Ib 0 Ob 1 e 0 p 1 g 0 chan 5.
		star: Erfolg. Trying to work around Kernel Pipe botch.

	before it continues. Since the star exit code in such a case is 0,
	we assume that this is a correct workaround and this case thus may
	be made completely silent in the future.

-	star: an even less frequent FIFO problem (occurs once every 50 million
	tries on fast multi CPU machines) has been identified. Star reports a
	hard EOF on input even though the complete file with logical EOF has
	been read and there is still input to process. In order to debug this
	problem a debug message has been added to the code.

	With this debug message, it turned out, that this problem happened
	because a context switch occurred in the FIFO read process after it did
	see an empty FIFO and later, after the process was resumed, the
	following check for the FIFO_MEOF flag did see EOF. We now first check
	for the FIFO_MEOF flag and later for the amount of data inside as the
	FIFO as FIFO_MEOF is set after the FIFO content has been updated and
	thus a context switch is no longer able to cause a wrong assumption
	about the content of the FIFO.

	If you still see this, please send a report.

-	star: added support to print debug malloc statistics to better debug
	memory problems in star.

-	star: pathname.c:: free_pspace() now only frees the path buffer if it
	is != NULL

-	star: fixed a bug in the file create.c that caused star to incorrectly
	grow the path buffer by 2 bytes for every archived file. This caused
	star to constantly grow if a larger amount of files are archived and
	eat up all memory available to 32 bit processes if the archived
	filesystem is larger than approx. 1 TB.

-	star: If the path name now cannot be handled because of low memory,
	we print a warning that includes the text "out of memory".

-	star: Now checking whether open of /dev/null failed while running a
	compress pipe. This avoids a core dump on defective OS installations.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: props.c: Added a missing /* FALLTHROUGH */ comment. 

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: create.c: Add more comment for the CPIO CRC format handler to
	explain why the last instance if a series of hard links for a file
	needs to archive the data.

-	star: diff.c: added a filling fillbytes(&finfo, ...) to make sure that
	ACL pointers are initialized.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: Several /* NOTREACHED */ comments have been added to tell
	programs like coverity that after a NULL pointer check, there is no
	continuation of the program

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: extract.c: A if (path->ps_path == '\0') has been corrected to
	if (path->ps_path[0] == '\0') after a mktemp() call. This was a typo
	introduced with the new support for extremely long path names.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: extract.c An initalization for a struct pathstore has been
	moved to the front to verify that path.ps_path is always initialized.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: header.c: isgnumagic(&ptb->dbuf.t_vers) has been changed to
	isgnumagic(ptb->ustar_dbuf.t_magic) as it is a "ustar" structure
	that is going to be checked.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	star: some Cstyle changes

-	ved: Added a missing /* FALLTHROUGH */ comment. 

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	bsh / Bourne Shell / star: the function hop_dirs() no longer checks
	for p2 != NULL before calling *p2 = '/' as p2 has been granted
	to be != NULL from a break with strchr(p, '/') == NULL

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Added a missing /* FALLTHROUGH */ comment. 

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: added a range check for $OPTIND to tge getopts(1)
	implementation

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Removed a nonsense variable in expand() that caused
	Coverity not to understand that a directory was correctly closed()

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Added a paranoia comparison to make Coverity quiet.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: avoid to call catpath() with a NULL pointer for path

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: check the write() return code in io.c

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Added a paranopia check for "test -o" to make Coperity
	quiet.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Added (void) before fcntl() in hope to make Coverity
	quiet.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Removed dead code from readwc()

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: Cstyle changes to xec.c

-	Bourne Shell: "builtin -d ..." did access free()d memory.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: expand.c: added a check for fd == -1 to avoid calling
	openat() with that fd.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: func.c: added a check for fd == -1 to avoid calling
	read()/close() with that fd.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: jobs.c: enlarged a buffer to be of same size as
	numbuf[] to avoid a potential buffer overflow.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: print.c::prt_cntl() had a very old (AT&T) bug with
	printing byte sequences that get an error with mbtowc() and it
	did not print byte sequences correctly that refer to legal but
	"nonprintable" multi byte characters.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: xec.c: the reserved word "time" could cause to
	access uninitialized memory if the string in $TIMEFORMAT contains
	the format %J

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	Bourne Shell: xec.c: added a paranoia check on whether
	findnam("funcname") returns NULL even though the existence of a
	function with that name already has been verified via the hash service.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SunPro Make: Added a missing /* FALLTHROUGH */ comment. 

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: libcomobj: dodelt.c & idsubst.c: added a forgotten
	/* FALLTHRU */

-	SCCS: admin.c: added some /* NOTREACHED */ comments to tell Coverity
	that fatal() does not return.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: admin.c: The function adjust() did call strlen() on a non-null
	terminated string.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: admin.c: The function adjust() returned a local string array.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: bdiff.c: added some /* NOTREACHED */ comments to tell Coverity
	that fatal() does not return.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: prs.c: The getopt() string did not contain 'q' for the expected
	support to the -q option.

-	SCCS: prs.c The code for handling the -d option was indented but curly
	brackets have been missing.

	Thanks to Pavel Raiskup for poiting to a related Coverity message.

-	SCCS: unget.c: The getopt() string did not contain 'q' for the expected
	support to the -q option.

-	SCCS: sccs.c Cstyle changes

-	SCCS: sccs.c Added support for new separate macro string defintions
	that apply to the "new mode".

-	SCCS: sccs.c New code allows to specify -N options specific to
	macro stgring defitions.

-	SCCS: sccs.c New environment "SCCS_NOEXEC" allows to disable command
	execution for debugging purposes.

-	SCCS: New version timestamp for SCCS commands lists 2019

-	ved: Added a missing /* FALLTHROUGH */ comment. 

	Thanks to Pavel Raiskup for poiting to a related Coverity message.


-	SCCS TODO:

	-	Convert sccs.c to use -NSCCS in the back end programs instead
		of converting g-file names from the command line into s.file
		names in the frontend in order to forward s.file names to the
		backend programs. Doing it this way allows us to have the SCCS
		history files "off tree" in a separate directory tree starting
		with ".sccs/" at the "project root" directory, as the related
		code to convert path names is in libcomobj::bulkprepare().

	-	Add code to admin(1) and delta(1) to retrieve the list of
		new or modified files in order to have all important code
		for a "sccs commit" in a single program that does not need to
		deal with ARG_MAX limitations.

	-	Add code to admin(1), delta(1) and get(1) to maintain/understand
		the changeset file.

	-	Add code to libcomobj to understand the changeset file,
		which is needed in order to e.g. know the file names and file
		specific SIDs/state that correspond to a project global SID.

	-	Add code to all programs to maintain a lock (for the file
		$PROJECTHOME/.sccs/SCCS/s.changeset, that would be in the file
		$PROJECTHOME/.sccs/SCCS/z.changeset) to maintain a project
		global lock.

	-	Find/verify a complete transactional model that allows to repair
		complex changes to the set of files for a project that have
		been aborted in the middle. The current idea is to create the
		file $PROJECTHOME/.sccs/changeset with the deltas to the
		changeset during a complex update operation.

	-	Find a way to decide when to use SCCS v6 type history files.
		For the project mode, this is needed in order to prevent 
		historic SCCS implementations to believe they could modify
		files without knowing about project global locks.





-	Bourne Shell Missing features for POSIX compliance:

	- Support for $'...' quoting (this is not needed for the current
					version of POSIX but for the next POSIX
					version that will be named SUSv8).
					The development of SUSv8 will start in
					late 2016.

	We are now expecting the Bourne Shell to be fully POSIX compliant.

-	Bourne Shell further TODO list:

	-	Finish loadable builtin support.

	-	POSIX does not allow us to implement ". -h", so we will
		add a "source" builtin to be able to implement "source -h"

-	The following builtins (that are available in bsh) are still missing in
	the Bourne Shell:

	err			echo with output going to stderr
	glob			echo with '\0' instead of ' ' between args
	env			a builtin version of /usr/bin/env

	The following bsh intrinsics are still missing in the Bourne Shell:

	-			the restricted bsh has restriction features that
				are missing in the Bourne shell.

	-	source -h	read file into history but do not execute

	and probably more features not yet identified to be bsh unique.



Author:

Joerg Schilling
D-13353 Berlin
Germany

Email: 	joerg@schily.net, joerg.schilling@fokus.fraunhofer.de

Please mail bugs and suggestions to me.
