2016-03-18  Sean Hefty  <sean.hefty@intel.com>

	release v1.1

	rstream: Allow user to set inline size
	Adjust the default inline size from 384 down to 64, so that more
	devices will be supported by default.  Add an option to allow
	the user to specify a different value.

2015-10-06  Hal Rosenstock  <hal@dev.mellanox.co.il>

	[librdmacm] examples/r[io]stream.c: Better handling of rpoll errors during client async rconnect
	Rather than silently exiting when connection is refused by server,
	print error message (Connection refused) during async rconnect at client.

	[librdmacm] rstream.c: Use proper define for ai_flags before getaddrinfo call in client_connect
	even though both have same value

	[librdmacm] examples: Use gai_strerror rather than perror for [rdma_]getaddrinfo failures
	[rdma_]getaddrinfo error codes are decoded by gai_strerror (and not
	set in errno) so replace perror calls following these failed calls.

2015-08-31  Sean Hefty  <sean.hefty@intel.com>

	Add parens to sizeof operator to clarify usage
	Convert sizeof *ptr -> sizeof(*ptr) to avoid confusion.  This
	is in response to the (false) bug report 2568.

2015-06-02  Sean Hefty  <sean.hefty@intel.com>

	v1.0.21

2015-05-29  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Delay initializing buffers until the inline size is known
	The qib HCA ignores the requested max_inline_size on input and
	instead returns the supported value on output.  As a result,
	the default inline size requested by rsockets is ignored, with 0
	being returned.  The code catches this after it creates the QP, but
	has already initialized its data buffers prior to creating the QP.

	The result is that the inline size used in rs_init_bufs() is larger
	than that supported by the qib device.  This causes a failure when
	attempting to update available receive buffer space.  The registered
	data buffer for the credit message is smaller than what is needed.

	Work-around this issue by delaying the initialization of the data
	buffers until after the QP has been created and the real size of
	the inline data is known.

2015-04-28  Sean Hefty  <sean.hefty@intel.com>

	Remove prints to stderr
	The library should just fail operations with ENODEV, rather than
	printing to stderr.  Printing can result in applications failing,
	or displaying incorrect error messages when no verb devices are
	actually present.

2015-04-09  Sean Hefty  <sean.hefty@intel.com>

	v1.0.20

2015-03-26  Ilya Nelkenbaum  <ilyan@mellanox.com>

	cma: Workaround for rdma_ucm kernel bug
	For certain new kernels, IB_QP_SMAC bit is erroneously
	set in QP attribute mask. This workaround turns
	off that bit. It allows SSA connections (AF_IB)
	to work. Without this workaround, there are issues
	on the client side.

	Kernel patch which caused issue is commit dd5f03b
	IB/core: Ethernet L2 attributes in verbs/cm structures

	Kernel patch to fix this in upstream:

	commit c2be9dc0e0fa59cc43c2c7084fc42b430809a0fe
	Author: Ilya Nelkenbaum <ilyan@mellanox.com>
	Date:   Thu Feb 5 13:53:48 2015 +0200

	    IB/core: When marshaling ucma path from user-space, clear unused fields

	    When marshaling a user path to the kernel struct ib_sa_path, we need
	    to zero smac and dmac and set the vlan id to the "no vlan" value.

	    This is to ensure that Ethernet attributes are not used with
	    InfiniBand QPs.

	    Fixes: dd5f03beb4f7 ("IB/core: Ethernet L2 attributes in verbs/cm structures")
	    Signed-off-by: Ilya Nelkenbaum <ilyan@mellanox.com>
	    Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
	    Signed-off-by: Roland Dreier <roland@purestorage.com>

	The fix was pushed to stable 3.14, 3.18 and 3.19 versions.

2015-03-26  Sasha Kotchubievsky  <sashakot@mellanox.com>

	rsocket; Set POLLERR bit in revent in case of rs_connect_error
	In non-blocking mode, on server side accept returns a new data
	socket and select/poll should indicate when the socket is ready
	for reading/writing.

	This fix allows an application to receive an error in case of
	connection failure.

2015-03-25  Hal Rosenstock  <hal@dev.mellanox.co.il>

	[librdmacm] rstream.c: Add missing binding to source address in client_connect
	This is needed for IPv6 connections.

2015-02-11  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Fix race in indexer map
	Although insertions and removals of rsockets are protected
	against accesses to the index map, when reading the map using
	a non-rsocket (i.e. normal fd), the reading of the map may
	overlap with the removal of an rsocket.  This can result in
	accessing freed memory.

	We can avoid this by not freeing the memory when rsockets
	no longer reference an index array.  This ensures that the
	memory is valid, and protects against reading the memory without
	adding locking into the read path.

	Problem reported by: Sasha Kotchubievsky <sashakot@mellanox.com>

2015-02-10  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Fix F_SETFL to use an int rather than long
	The flags are specified as an int, not a long.  The
	rsocket code handles the flag incorrectly.

2015-02-06  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Fix setting flags in rfctl
	The rfcntl() call to set rsocket flags merely OR's in
	the updated flags with the existing ones, rather than
	replacing them.  Also, it does not handle setting an
	rsocket from nonblocking mode back to blocking mode.

2015-01-22  Steve Wise  <swise@opengridcomputing.com>

	rping: create persistent server threads in DETACHED state
	Since the persistent server threads aren't joined, they must be created in
	the DETACHED state or resources will not be cleaned up when they exit.
	This results in pthread_create() failures after thousands of rping
	instances are run against a persistent server.

	Also check the return from all calls to pthread_create() so we don't
	ignore a thread creation failure.

2014-11-14  Hariprasad S  <hariprasad@chelsio.com>

	rping: ignore flushed completions
	Based on original work by Steve Wise <steve@opengridcomputing.com>

	rping: Fixes race, where ibv context was getting freed before memory was deregistered
	While running rping as a client without server on the other end,
	rping_test_client fails and the ibv context was getting freed
	before memory was deregistered. This patch fixes it.

2014-10-08  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Return size of received data in case of error
	If an error occurs processing rrecv(), we return the error.
	But if we have data, we should return the size of the
	available data instead.

2014-09-29  Sreedhar Kodali  <srkodali@linux.vnet.ibm.com>

	While waiting for a completion event, rsocket state is incorrectly set to error when interrupted.  Instead, the caller of get completion event should decide what to do with it based on errno.  The fix is do not change the state to rs_error when errno is EINTR inside get completion event.
	Reviewed-by: Pradeep Satyanarayana <pradeeps@linux.vnet.ibm.com>

	Perform completion event acknowledgments in batches instead of individually to minimze locking overheads.  Size of the completion queue decides the size of a batch.

2014-09-25  Sreedhar Kodali  <srkodali@linux.vnet.ibm.com>

	rsockets: Add fine grained interception mechanism for preload library
	By default the R-Sockets pre-loading library intercepts all
	the stream and datagram sockets belonging to a launched
	program processes and threads.

	However, distributed application and database servers may
	require fine grained interception to ensure that only the
	processes which are listening for remote connections on the
	RDMA transport need to be enabled with RDMA while remaining
	can continue to use TCP as before.  This allows proper
	communication happening between various server components locally.

	A configuration file based mechanism is introduced to facilitate
	this fine grained interception mechanism.  As part of preload
	initialization, the configuration file is scanned and an
	in-memory record store is created with all the entries found.
	When a request is made to intercept a socket, its attributes
	are cross checked with stored records to see whether we
	should proceed with rsocket switch over.

	Note: Right now, the fine grained interception mechanism is
	enabled only for newly created sockets.  Going forward,
	this can be extened to select connections based on the
	specified host/IP addresses and ports as well.

	"preload_config" is the name of the configuration file which
	should exist in the default configuration location
	(usually the full path to this configuration file is:
	<install-root>/etc/rdma/rsocket/preload_config)
	of an installed rsocket library.

	The sample format for this configuration file is shown below:

	# Sample config file for preloading in a program specific way
	#
	# Each line entry should have the following format:
	#
	#   program domain type protocol
	#
	# where,
	#
	# program    - program or command name (string without spaces)
	# domain     - the socket domain: AF_INET / AF_INET6 / AF_IB
	# type       - the socket type: SOCK_STREAM / SOCK_DGRAM
	# protocol   - the socket protocol: IPPROTO_TCP / IPPROTO_UDP
	#
	# The wildcard value of '*' is supported for any
	#
	# Note:
	#  Lines beginning with '#' character are treated as comments.

	Reviewed-by: Pradeep Satyanarayana <pradeeps@linux.vnet.ibm.com>

2014-09-04  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Support calling listen multiple times on same rsocket
	Standard sockets allows an application to call listen() multiple
	times on the same socket without error.  This allows a multi-threaded
	app to call listen from all threads.

	rsockets will fail the second listen call.  Modify the behavior to
	match standard sockets.

	Problem reported by: Sreedhar Kodali <srkodali@linux.vnet.ibm.com>

2014-09-04  Sasha Kotchubievsky  <sashakot@mellanox.com>

	rsocket: Index map item is cleaned before it is used in iomapping cleanup
	rs_free function clears index map item corresponding to the roscket
	(in idm_clear called from rs_remove) and then uses it in
	iomapping cleanup (in riounmap called from rs_free_iomappings).

2014-08-18  Sean Hefty  <sean.hefty@intel.com>

	Renumber 1.0.19-1 release to 1.0.19.1

2014-08-08  Sean Hefty  <sean.hefty@intel.com>

	Merge branch 'dev'

	Release 1.0.19-1 hotfix

2014-08-04  Sean Hefty  <sean.hefty@intel.com>

	indexer: Include errno.h directly

2014-07-28  Ilya Nelkenbaum  <ilyan@mellanox.com>

	rsocket: Segmentation fault fix in case of multiple connections
	In case of more than 16 rsocket connections
	are established, "svc->rss" buffer is reallocated
	with more memory. Index 0 is reserved for the service's
	communication socket, and this is not taken in count
	when data is copied from old buffer location to
	new one.

2014-07-23  Sean Hefty  <sean.hefty@intel.com>

	udpong: Fix client_recv error check
	We only want to report an error if it's not EGAIN.  The if
	statement is reversed.  Correct it.

2014-07-17  Sean Hefty  <sean.hefty@intel.com>

	Release 1.0.19

2014-07-16  Sean Hefty  <sean.hefty@intel.com>

	riostream: Only verify last data transfer
	Data verification will fail when running the bandwidth
	tests or the transfer count is > 1.  The issue is that
	subsequent writes by the initiator side will overwrite
	the data in the target buffer before the receiver can
	verify that it is correct.

	To fix this, only verify that the data in the buffer
	is correct after the last transfer has completed.

2014-07-07  Sean Hefty  <sean.hefty@intel.com>

	Revert "Revert "rsocket: Change keepalive to 0-byte RDMA write""
	This reverts commit a34703c53259845dd20450a87eb6747030e23e8b.

	0-byte RDMA writes appears to be working correctly with
	HCAs from 2 different vendors.  The original problem that
	was reported turned out to be a user error.

2014-07-03  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Update correct rsocket keepalive time
	When the keepalive time of an rsocket is updated, the
	updated information is forwarded to the keepalive service
	thread.  However, the thread updates the time for the
	wrong service as shown:

	tcp_svc_timeouts[svc->cnt] = rs_get_time() + msg.rs->keepalive_time;

	The index into tcp_svc_timeouts should correspond to the
	rsocket being updated, not the last one in the list.

2014-07-03  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Fix removing rsocket from service thread
	When removing an rsocket from a service thread, we replace
	the removed service with the one at the end of the service list.
	This keeps the array tightly packed.  However, rs_svc_rm_rs
	decrements the rsocket count before doing the swap.  The result
	is that the entry at the end of the list gets dropped off.
	Defer decrementing the count until the swap has been made.

	In this case, the cnt value is a valid index into the array,
	because we start at index 1.  Index 0 is used internally by
	the service thread.

2014-07-03  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Fix crash resulting from keepalive timeout
	The following crash was reported by Hal Rosenstock,
	<hal@mellanox.com>, with keepalive enabled.  The crash
	occurs in the keepalive thread attempting to send a
	keepalive message.

	report:
	Program received signal SIGSEGV, Segmentation fault.
	[Switching to Thread 0x7fffecf08700 (LWP 6013)]
	rs_post_write (rs=<value optimized out>, sgl=0x0, nsge=0, wr_data=3758096385,
	    flags=0, addr=0, rkey=0) at src/rsocket.c:1660
	1660            return rdma_seterrno(ibv_post_send(rs->cm_id->qp, &wr, &bad));
	Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64
	(gdb)
	(gdb) p/x rs
	$1 = value has been optimized out

	So I added in the following to debug:
	1660    if (rs == NULL)
	1661    abort();
	1662    if (rs->cm_id == NULL)
	1663    abort();
	1664    if (rs->cm_id->qp == NULL)
	1665    abort();
	1666            return rdma_seterrno(ibv_post_send(rs->cm_id->qp, &wr, &bad));
	1667    }

	And saw in gdb:

	Program received signal SIGABRT, Aborted.
	[Switching to Thread 0x7fffecf08700 (LWP 8096)]
	0x00000030d50328a5 in raise () from /lib64/libc.so.6
	Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.107.el6.x86_64
	(gdb)
	(gdb) bt
	#0  0x00000030d50328a5 in raise () from /lib64/libc.so.6
	#1  0x00000030d5034085 in abort () from /lib64/libc.so.6
	#2  0x00007ffff057fe23 in rs_post_write (rs=<value optimized out>, sgl=0x1fa0,
	    nsge=6, wr_data=4294967295, flags=0, addr=0, rkey=0) at src/rsocket.c:1665
	#3  0x00007ffff058193d in tcp_svc_send_keepalive (arg=0x7ffff0789f20)
	    at src/rsocket.c:4245
	#4  tcp_svc_run (arg=0x7ffff0789f20) at src/rsocket.c:4279
	#5  0x00000030d5807851 in start_thread () from /lib64/libpthread.so.0
	#6  0x00000030d50e890d in clone () from /lib64/libc.so.6
	(gdb) fr 2
	#2  0x00007ffff057fe23 in rs_post_write (rs=<value optimized out>, sgl=0x1fa0,
	    nsge=6, wr_data=4294967295, flags=0, addr=0, rkey=0) at src/rsocket.c:1665
	1665    abort();

	So qp is NULL somehow...
	:end report

	There is an issue if an rsocket is closed without going through
	the rshutdown.

	int rshutdown(int socket, int how)
	{
		...
		if (rs->opts & RS_OPT_SVC_ACTIVE)
			rs_notify_svc(&tcp_svc, rs, RS_SVC_REM_KEEPALIVE);

	We remove the rsocket from the keepalive thread in rshutdown.

	int rclose(int socket)
	{
		...
			if (rs->state & rs_connected)
				rshutdown(socket, SHUT_RDWR);
		...
		rs_free(rs);

	rclose will call shutdown only if we're connected.  However, if the
	keepalive failed, the socket will be in an error state.  So,
	no call to rshutdown, which will leave the freed rsocket on
	the keepalive thread's list.

	The fix is to to have rclose remove an rsocket from being processed
	by a service thread if it is still active.

2014-07-02  Sean Hefty  <sean.hefty@intel.com>

	example/rdma_xclient/server: Update XRC support in sample programs
	Update rdma_xclient and rdma_xserver sample programs to test
	XRC data transfers.

	rdmacm: Update addrinfo with XRC support
	Remove internal defines, and use libibverbs exported values
	instead.

	rdmacm: Add support for XRC QPs
	Export a new extended create QP call.  Add support for XRC
	QPs.

	rdmacm: Add support for allocating XRC SRQs
	Add extended SRQ creation call, to support allocating
	XRC SRQs.  Use the rdma_cm_id qp type field to
	determine which type of SRQ should be allocated.

	rdmacm: Add functionality to allocate an XRCD
	XRC QPs and SRQs are associated by an XRC domain.  Provide a
	call to allocate an XRCD, similar to how the rdmacm allocates
	a PD for the user.

2014-07-02  Sean Hefty  <sean.hefty@intel.com>

	build: Add build support for XRC
	Modify autotools to check for and require a libibverbs
	version that includes XRC and extension support.

	Remove any code used to support older versions of
	libibverbs.

2014-07-01  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Use SRQ in rdma_create_qp
	If an application has allocated an SRQ on an rdma_cm_id, use
	it when creating a QP.

2014-06-25  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Remove NULL checks after calling alloca
	alloca doesn't return a NULL pointer on failure.

2014-06-21  Sean Hefty  <sean.hefty@intel.com>

	Revert "rsocket: Change keepalive to 0-byte RDMA write"
	This reverts commit 0f2c76e81ecf1470cf152600c08c421e7e82b00e.

	Testing has shown that this does not always result in the
	keep-alive message working correctly, such that a broken
	connection is reported as having failed.  The reason for this
	behavior is unknown, but revert the patch until the issue has
	been resolved.

2014-06-20  Hal Rosenstock  <hal@dev.mellanox.co.il>

	librdmacm: In ucma_convert_path, fix selector values
	Intent is for the selectors to be equal to (exactly) rather than less than.
	Selector for exactly is value of 2 rather than 1.

	rsocket: Add support for RDMA_ROUTE option in rgetsockopt
	Create as many ibv_path_data structs from the RDMA route
	ibv_sa_path_rec struct for the rsocket based on how
	many fit into the supplied buffer.

2014-06-18  Sean Hefty  <sean.hefty@intel.com>

	Merge branch 'dev'

2014-06-18  Susan K. Coulter  <markus@cj-fe1.lanl.gov>

	rsocket: Change keepalive to 0-byte RDMA write

2014-06-18  Doug Ledford  <dledford@redhat.com>

	rdma_server: handle IBV_SEND_INLINE correctly
	Not all RDMA devices support IBV_SEND_INLINE.  At least some of those
	that don't will ignore the flag passed to rdma_post_send and attempt to
	send the command by using an sge entry instead.  Because we don't
	register the send memory, this fails.  The proper way to deal with the
	fact that IBV_SEND_INLINE is not guaranteed is to check the returned
	value in our cap struct to see if we have support for inline data, and
	if not, fall back to non-inline sends and to register the send memory
	region.

	rdma_client: handle IBV_SEND_INLINE correctly
	Not all RDMA devices support IBV_SEND_INLINE.  At least some of those
	that don't will ignore the flag passed to rdma_post_send and attempt to
	send the command by using an sge entry instead.  Because we don't
	register the send memory, this fails.  The proper way to deal with the
	fact that IBV_SEND_INLINE is not guaranteed is to check the returned
	value in our cap struct to see if we have support for inline data, and
	if not, fall back to non-inline sends and to register the send memory
	region.

	rdma_server: use perror, unwind allocs on failure
	Our main test function prints out errno directly, which is hard to read
	as it's not decoded at all.  Instead, use perror() to make failures more
	readable.  Also redo the failure flow so that we can do a simple unwind
	at the end of the function and just jump to the right unwind spot on
	error.

	rdma_client: use perror, unwind allocs on failure
	Our main test function prints out errno directly, which is hard to read
	as it's not decoded at all.  Instead, use perror() to make failures more
	readable.  Also redo the failure flow so that we can do a simple unwind
	at the end of the function and just jump to the right unwind spot on
	error.

2014-06-18  Doug Ledford  <dledford@redhat.com>

	cmtime: rework program to be multithread
	When using very large numbers of connections (10,000 was in use here),
	we ran into a problem where when we resolved a performance problem in
	the kernel cma.c code, we suddenly developed a new problem.  That new
	problem turned out to be the fact that with the underlying kernel issue
	resolved, 10,000 connect requests would flood the server side of the
	test and the cmtime application would respond as quickly as possible.
	However, the client side would not bother to check any of the returns
	until after having sent all 10,000 connect requests.  When the kernel
	had a serializing performance problem, this was OK.  When it was fixed,
	this caused a general slowdown in connect operations due to overruns in
	the event processing.  This patch causes the client side to fire off
	threads that will handle responses to connect requests as they come in
	instead of allowing them to backlog uncontrollably.  Times for a 10,000
	connect run changed from this:

	[root@rdma-dev-01 ~]# more
	3.12.0-rc1.cached_gids+optimized_connect+trimmed_cache+.output
	ib1:
	step              total ms     max ms     min us  us / conn
	create id    :       46.64       0.10       1.00       4.66
	bind addr    :       89.61       0.04       7.00       8.96
	resolve addr :       50.63      26.18   23976.00       5.06
	resolve route:      565.44     538.77   26736.00      56.54
	create qp    :     4028.31       5.70     326.00     402.83
	connect      :    50077.42   49990.49   90734.00    5007.74
	disconnect   :     5277.25    4850.35  380017.00     527.72
	destroy      :       42.15       0.04       2.00       4.21

	ib0:
	step              total ms     max ms     min us  us / conn
	create id    :       34.82       0.04       1.00       3.48
	bind addr    :       25.94       0.02       1.00       2.59
	resolve addr :       48.18      25.01   22779.00       4.82
	resolve route:      501.28     476.26   25071.00      50.13
	create qp    :     3274.12       6.05     257.00     327.41
	connect      :    55549.64   55490.32   62150.00    5554.96
	disconnect   :     5263.64    4851.18  375628.00     526.36
	destroy      :       47.20       0.07       2.00       4.72

	to this:

	[root@rdma-dev-01 ~]# more
	3.12.0-rc1.cached_gids+optimized_connect+trimmed_cache+-fixed-cmtime.output
	ib1:
	step              total ms     max ms     min us  us / conn
	create id    :       34.45       0.08       1.00       3.44
	bind addr    :       88.41       0.04       7.00       8.84
	resolve addr :       33.59       4.65     612.00       3.36
	resolve route:      618.68       0.61      97.00      61.87
	create qp    :     4024.03       6.30     341.00     402.40
	connect      :     6983.35    6886.33    8509.00     698.33
	disconnect   :     5066.47     230.34     831.00     506.65
	destroy      :       37.02       0.03       2.00       3.70

	ib0:
	step              total ms     max ms     min us  us / conn
	create id    :       42.61       0.14       1.00       4.26
	bind addr    :       27.05       0.03       2.00       2.70
	resolve addr :       40.65      10.73     869.00       4.06
	resolve route:      626.75       0.60     103.00      62.68
	create qp    :     3334.50       6.48     273.00     333.45
	connect      :     6310.29    6251.59   13298.00     631.03
	disconnect   :     5111.12     365.87     867.00     511.11
	destroy      :       36.57       0.02       2.00       3.66

	with this patch.

2014-06-18  Hal Rosenstock  <hal@mellanox.com>

	rsocket: Use malloc instead of calloc
	No need to clear allocated memory as immediately followed by
	memcpy which covers the allocated memory.

2014-05-27  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Update rdma_accept man page
	Document NULL conn_param parameter for rdma_accept.

2014-05-23  Sean Hefty  <sean.hefty@intel.com>

	indexer: Free index_map resources when cleared
	Free memory allocated for index map entries when they are no
	longer in use.  To handle this, count the number of entries
	stored by the index map item arrays and release the arrays when
	no items are being tracked.

	This reduces valgrind noise.

	Problem reported by: Hannes Weisbach <hannes_weisbach@gmx.net>

2014-04-30  Patrick MacArthur  <pmacarth@iol.unh.edu>

	rstream: fix "-T resolve" detection

2014-04-30  shamir rabinovitch  <shamir.rabinovitch@oracle.com>

	librdmacm: Fix verbs leak due to reentrancy issue
	Any call to ucma_init_device must be done under lock.

2014-04-30  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Relax requirement for minimal inline data
	Inline data support is optional.  Allow rsockets to work
	with devices that do not support inline data, provided
	that they do support RDMA writes with immediate data.
	This allows rsockets to work over Intel TrueScale HCA.

	Patch derived from work by: Amir Hanania

2014-04-30  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Modify when control messages are available
	Rsockets currently tracks how many control messages (i.e.
	entries in the send queue) that are available using a
	single ctrl_avail counter.  Seems simple enough.

	However, control messages currently require the use of
	inline data.  In order to support control messages that
	do not use inline data, we need to associate each
	control message with a specific data buffer.  This will
	become easier to manage if we modify how we track when
	control messages are available.

	We replace the single ctrl_avail counter with two new
	counters.  The new counters conceptually treat control
	messages as if each message had its own sequence number.
	The sequence number will then be able to correspond to
	a specific data buffer in a follow up patch.

	ctrl_seqno will be used to indicate the current control
	message being sent.  ctrl_max_seqno will track the
	highest control message that may be sent.

	A side effect of this change is that we will be able to
	see how many control messages have been sent.  This also
	separates the updating of the control count on the
	sending  side, versus the receiving side.

2014-04-30  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Dedicate a fixed number of SQEs for control messages
	The number of SQEs allocated for control messages is set
	to 1 of 2 constant values (either 4 or 2).  A default
	value is used unless the size of the SQ is below a certain
	threshold (16 entries).  This results in additional code
	complexity, and it is highly unlikely that the SQ would
	ever be allocated smaller than 16 entries.

	Simplify the code to use a single constant value for the
	number of SQEs allocated for control messages.  This will
	also help in subsequent patches that will need to deal
	with HCAs that do not support inline data.

2014-04-30  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Check max inline data after creating QP
	The ipath provider will ignore the max_inline_size
	specified as input into ibv_create_qp and instead
	return the size that it supports (which is 0) on
	output.

	Update the actual inline size returned from create QP,
	and check that it meets the minimum requirement for
	rsockets.

2014-04-30  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Make ucma_init_all static

2014-04-09  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Support lazy initialization
	librdmacm currently opens a device context per configured HCA. This is
	usually done in rdma_create_event_channel() or first time whenever
	ucma_init() is called. If a process is only going to use one of the
	configured HCAs/RDMA IPs then the remaining device contexts are not
	used/required. Opening a device context on each device apriori limits the
	maximum number of processes that can be supported on a node to the maximum
	number of open context supported per HCA regardless of number of HCAs present
	in the system.

2014-03-06  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Fix sbuf_bytes_avail counter 'overrun' with iwarp
	Reported-by: Jonas Pfefferle1 <JPF@zurich.ibm.com>

	"The problem is that on the client side sbuf_bytes_avail overflows
	in rs_poll_cq.  And from what I debugged so far there are 2
	completions for every send and this is because I use iWarp hardware
	which does not support write with immediate so there is one completion
	for the write and one for the send (both go into the default case
	and add the length to sbuf_bytes_avail)."

	To avoid the issue, we flag send message operations that are used
	in place of immediate data.  Other send message operations are
	not affected.  The completion code can then check whether the
	completion is for a send message which was paired with an RDMA
	write transaction and adjust the behavior accordingly.  Additionally,
	such send messages only carry the opcode in their WR_ID, with the
	data portion zeroed.  This avoids adding the length value twice.

2014-03-05  Hal Rosenstock  <hal@mellanox.com>

	riostream: Add AF_IB support
	Allow the user to specify GID addresses (AF_IB) with riostream

	rsocket: Return EBADF on bad rsocket fd
	Eliminates potential seg faults when passed an invalid rsocket.

2014-03-05  Sean Hefty  <sean.hefty@intel.com>

	man/rsocket: Enhance riomap documentation
	Document that the user must set IOMAPSIZE in order to
	use the riomap call.

2014-01-27  Sean Hefty  <sean.hefty@intel.com>

	librdmacm 1.0.18

	udaddy: Remove support for port space IB
	UD support for the IB port space requires that the application
	use rdma_create_ep, rather than rdma_create_id.  However, using
	rdma_create_ep results in address and route resolution being
	performed synchronously as part of the rdma_create_ep call.
	Since udaddy is an example, we want to show how it can be used
	with asynchronous events.  So, rather than update udaddy to
	use rdma_create_ep in order to support the IB port space, it
	would be better to remove that support.

2014-01-17  Susan K. Coulter  <markus@cj-fe2.lanl.gov>

	rsocket: Add keepalive logic
	Actually send and receive keepalive messages if keepalive is
	enabled on an rsocket.

2013-12-04  Or Gerlitz  <ogerlitz@mellanox.com>

	librdmacm: Add directives on binding to IPv6 any address to man pages
	Explain how to bind to IPv6 any address in the man pages for the examples

2013-11-26  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Check 'init' under mutex
	ucma_ib_init() does a quick check that access to ibacm has
	been initialized.  This check is done outside of the
	acm_lock mutex.  We need to check init again inside of
	holding the mutex to ensure that we don't run the
	initialization code twice.

2013-11-18  Sean Hefty  <sean.hefty@intel.com>

	rping: Fix server reporting error on exit
	Commit e57196c71ddd850e14f3e66355f02786e4914f72
	rping: added checks to the return values functions
	resulted in the rping server always reporting that
	it failed.  Fix this by only failing in the case of
	an unexpected termination, and not the result of
	the client completing.

2013-11-11  Sean Hefty  <sean.hefty@intel.com>

	Retrieve SGID after calling rdma_bind_addr
	A change was made to rdma_bind_addr when AF_IB is enabled
	to only retrieve the resulting bound address.  Previously,
	rdma_bind_addr would retrieve the corresponding SGID as
	well.  This breaks some apps which were checking the
	SGID after binding to an IP address.  Revert to the
	previous behavior of also retrieving the SGID after
	calling rdma_bind_addr.

	Tested-by: Christoph Lameter <cl@linux.com>

2013-11-07  Guy Shapiro  <guysh@mellanox.com>

	librdmacm: Some fixes to man pages
	Fix the man pages of rdma_destroy_ep & rdma_destroy_qp to the correct return value (void).

2013-11-07  Hal Rosenstock  <hal@dev.mellanox.co.il>

	[librdmacm] Makefile.am: Add missing riostream man page to man_MANS

2013-10-30  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Handle race between rshutdown and rpoll
	Multi-threaded applications which call rpoll and rshutdown
	simultaneously can hang.  Ceph developers reported an issue
	with the rsocket implementation.  Ceph calls rpoll in
	one thread, and while that thread is blocked in rpoll,
	a second thread may cann rshutdown on the socket.  In
	normal sockets, this results in the poll call unblocking
	(since a call to read on the socket will no longer block).
	however, rsockets does not free the thread blocked on the
	rpoll call.

	To fix this, we add some additional state checking to
	protect against threads calling rpoll and rshutdown
	simultaneously.  We also have the rshutdown call
	transition the QP into an error state.  This causes all
	posted receives to complete as flushed, which results
	in unblocking the thread in rpoll (to process the flushed
	receives).

2013-09-21  Hal Rosenstock  <hal@dev.mellanox.co.il>

	[librdmacm] man/rstream.1: Update man page to be consistent with rstream -h

	[librdmacm] rstream.c: Indicate when specified address family is unknown

	[librdmacm] man/rdma_create_id.3: Add RDMA_PS_IB port space description

2013-08-27  Yan Droneaud  <ydroneaud@opteya.com>

	examples: Add cmtime to .gitignore

2013-08-23  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Update rsocket man page
	Update fork support and RDMA_ROUTE socket option.

2013-08-22  Sean Hefty  <sean.hefty@intel.com>

	cmtime: Add retry support for address and route resolution

	cmtime: Allow user to specify timeout values

	cmtime: Add ability to time rdma_bind_addr calls

2013-08-05  Sean Hefty  <sean.hefty@intel.com>

	cmtime: Add example program that times rdma cm calls
	cmtime is a new sample program that measures how long it
	takes for each step in the connection process to complete.
	It can be used to analyze the performance of the various
	CM steps.

2013-08-03  Sean Hefty  <sean.hefty@intel.com>

	rstream: Use rsocket option to set route directly
	If we're using GID addressing, rdma_getaddrinfo can return
	routing data directly.  Add an option for the user to
	indicate that rdma_getaddrinfo should be called in place of
	getaddrinfo.  And if routing data is available, call
	rsetsockopt to set the route.

	This helps test rsockets when ibacm and AF_IB support are
	available.

2013-08-03  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Return 0 on success for SOL_RDMA options
	The processing of SOL_RDMA does not set the return value in
	the case of successfully handled options.

	rsockets: Add ability to set the IB route directly
	Add an RDMA specific rsocket option that allows the user
	to program the RDMA route directly.  This is useful
	for apps that have path record data available, e.g. from
	ibacm.

	examples: Add support for native IB addressing to samples
	Allow the user to specify GID addresses (AF_IB) into
	udaddy and rstream.

2013-07-31  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Support native IB addressing on connected rsockets
	Update rsockets to support AF_IB addresses on connected rsockets.
	Support for datagram rsockets is more difficult as a result of
	using real UDP sockets for QP resolution, so that support is
	deferred.  For connected sockets, we need to update internal
	checks to handle AF_IB.

2013-07-31  Bart Van Assche  <bvanassche@acm.org>

	[4/4] Declare 'server_port' as an unsigned variable
	Change the data type of the 'server_port' variable from signed to
	unsigned such that the cast in the fscanf() call can be removed.

2013-07-31  Bart Van Assche  <bvanassche@acm.org>

	[3/4] rsocket: Remove the unused variable 'ret'
	The variable 'ret' is assigned a value but that value is never used.
	This triggers the following compiler warning:

	src/rsocket.c:3720:9: warning: variable 'ret' set but not used [-Wunused-but-set-variable]

	Hence remove this variable.

2013-07-31  Bart Van Assche  <bvanassche@acm.org>

	[2/4] cma: Remove the unused variable 'id_priv'
	The variable 'id_priv' is assigned a value but is never used.
	This triggers the following compiler warning:

	src/cma.c:1178:25: warning: variable 'id_priv' set but not used [-Wunused-but-set-variable]

	Hence remove this variable.

2013-07-31  Bart Van Assche  <bvanassche@acm.org>

	[1/4] acm: Remove the unused variable 'pri_path'
	The variable 'pri_path' is assigned a value but is never used.
	This triggers the following compiler warning:

	src/acm.c:301:26: warning: variable 'pri_path' set but not used [-Wunused-but-set-variable]

	Hence remove this variable.

2013-07-18  Sean Hefty  <sean.hefty@intel.com>

	init: Remove USE_IB_ACM configuration option
	When the librdmacm is configured, it sets the USE_IB_ACM option
	if infininband/acm.h is found.  We can remove this option with
	very little overhead, which would allow a user to install
	ACM after installing the librdmacm, and the librdmacm would be
	able to make use of ACM.

2013-07-18  Sean Hefty  <sean.hefty@intel.com>

	acm: Define needed ACM protocol messages
	The librdmacm needs message definitions used to communicate
	with the ibacm.  It currently pulls these from infiniband/acm.h,
	which is installed by ibacm.  This creates an install order
	dependency on ibacm.  However, work on the scalable SA has
	the ibacm using the librdmacm (via rsockets) for communication
	between the different SSA components.

	To resolve this issue, have the librdmacm define the message
	structures that it needs to communicate with ibacm.  The
	librdmacm already defines some ACM messages through configuration
	checks.  We just expand that capability, which isolates the librdmacm
	package from the ibacm package.

2013-07-17  Sean Hefty  <sean.hefty@intel.com>

	cmatose: Allow user to specify address format
	Provide an option for the user to indicate the type of
	addresses used as input.  Support hostname, IPv4, IPv6,
	and GIDs.

2013-07-17  Yann Droneaud  <ydroneaud@opteya.com>

	Remove executable mode bit on text files
	Source code and man page should not be executable.

2013-07-17  Yann Droneaud  <ydroneaud@opteya.com>

	Open files with "close on exec" flag
	File opened by librdmacm are not supposed to be inherited across
	exec*(), most of the files are of no use for another program, and
	others cannot be used without the associated memory mapping.

	This patch changes fopen() open() and socket() to always set
	close on exec flag.

	This patch also add checks to configure to guess if fopen() supports
	"e" flag. If O_CLOEXEC and SOCK_CLOEXEC are supported, fopen() should
	support "e". If not supported, its discarded according to POSIX. Many
	operating systems have support for fopen("e").

	You might find more information about close on exec in the following articles:

	- "Excuse me son, but your code is leaking !!!" by Dan Walsh
	  http://danwalsh.livejournal.com/53603.html

	- "Secure File Descriptor Handling" by Ulrich Drepper
	  http://udrepper.livejournal.com/20407.html

	Note: this patch won't set close on exec flag on file descriptors
	created by the kernel for completion channel and such.
	This is addressed by another kernel patch.

2013-07-17  Yann Droneaud  <ydroneaud@opteya.com>

	Add .gitignore rules
	Add the list of files/patterns to be exclueded from git status output.
	Additionally it will prevent such files/patterns to be added and committed.

2013-07-17  Yann Droneaud  <ydroneaud@opteya.com>

	configure: Use automake's option "subdir-objects"
	Following advice in "Autotool Mythbuster" [1], option subdir-objects
	can be used to have Makefiles create object files in the same
	directory than theirs source files.

	It reduces clobbering in the build directory.

	[1] "Autotool Mythbuster", by Diego Elio "Flameeyes" Petten`o
	http://www.flameeyes.eu/autotools-mythbuster/automake/nonrecursive.html

2013-07-17  Yann Droneaud  <ydroneaud@opteya.com>

	configure: Apply updates proposed by autoupdate
	'autoupdate' is a tool to help developer to update configure.ac.

	This patch applies a few fixes as suggested by autoupdate.

2013-07-17  Jeff Squyres  <jsquyres@cisco.com>

	autogen.sh: Use autoreconf in autogen.sh
	The old sequence of Autotools commands listed in autogen.sh is no
	longer correct.  Instead, just use the single "autoreconf" command,
	which will invoke all the Right Autotools commands in the correct
	order.

2013-07-17  Bart Van Assche  <bvanassche@acm.org>

	Makefile.am: Fix an automake warning
	Fix the following automake warning message:

	    Makefile.am:1: `INCLUDES' is the old name for `AM_CPPFLAGS' (or `*_CPPFLAGS')

	A quote from the automake manual:

	    INCLUDES
	        This does the same job as AM_CPPFLAGS (or any per-target _CPPFLAGS variable
	        if it is used). It is an older name for the same functionality. This
	        variable is deprecated; we suggest using AM_CPPFLAGS and per-target
	        _CPPFLAGS instead.

2013-07-17  Bart Van Assche  <bvanassche@acm.org>

	Add "foreign" option to AM_INIT_AUTOMAKE
	Switch to the modern form of the AM_INIT_AUTOMAKE macro and tell
	automake that the librdmacm package does not follow the GNU
	standards. This change makes it possible to use 'autoreconf' for the
	librdmacm package.

2013-05-03  Sean Hefty  <sean.hefty@intel.com>

	lib: Rename configure.in to configure.ac
	Update to latest autotools naming.

2013-04-13  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add support for iWarp
	iWarp does not support RDMA writes with immediate data.
	Instead of sending messages using immediate data, allow
	the rsocket protocol to exchange messages using sends.

	The rsocket protocol remains the same.  RDMA writes are
	used for data transfers, with send messages used to transfer
	rsocket protocol messages.

2013-04-12  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Merge usage of wr_id between stream and datagram svcs
	The rsocket data streaming and datagram services use different
	formats for the wr_id.  Although some differences are needed,
	we can make them more similar.  This will be useful when the
	wr_id is used for iwarp support, plus eliminates use of wr_id
	bits that aren't actually needed.

2013-03-06  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Release 1.0.17

2013-02-20  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rsocket: Fix resetting O_NONBLOCK after calling shutdown
	Shutdown switches an rsocket from nonblocking to blocking to
	ensure that all data has been sent.  After completing all
	transfers, it should switch back to nonblocking; this handles
	partial shutdown situations, where only half the connection
	is shut down.  However, the code uses the value of '1' to
	set the nonblocking flag, rather than O_NONBLOCK.  Fix this.

2013-02-05  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rstream: Reduce default transfer count
	1 million ping-pong transfers takes over 3 seconds to
	complete, and I'm impatient.  Reduce the default number of
	transfers for small messsages to speed up running
	performance tests, especially when running over slower
	connections, like TCP sockets or over a WAN.

2013-02-02  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Work-around kernel bug returning uid = 0
	Older kernels have a bug where it can report an event with the
	uid set to 0.  The librdmacm crashes when casting the uid to
	an rdma_cm_id and dereferencing the NULL pointer.

	There are a limited number of events where this can occur and
	in most cases it's safe to simply discard the event.  (This is
	what the kernel does anyway.)  However, it's possible for us
	to process an RDMA_CM_EVENT_ESTABLISHED event with the uid
	set to 0.  (See kernel commit 418edaaba96e58112b15c82b4907084e2a9caf42.)

	Although it's rare for this to occur, it does in fact happen
	in practice.  To work-around the kernel bug, when the uid of an
	established event is set to 0, we first try to locate the correct
	user space id based on related data before discarding the event.

2013-01-28  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Define ucma_ib_init when IB_ACM is disabled
	ucma_ib_init is only defined if IB_ACM is enabled, which is
	determined by looking for the infiniband/acm.h header file.
	Define ucma_ib_init when IB_ACM is disabled.

	Problem reportedy by Suresh Shelvapille <suri@baymicrosystems.com>

2013-01-22  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Update rsocket man page
	Update man page to include recently added rsocket options
	and undocumented configuration file.

2013-01-09  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Add support for existing UDP apps
	Support for existing UDP applications is done via the rspreload
	library.  However, when the preload library is loaded, socket
	calls used by rsockets get intercepted and converted into
	rsocket calls.

	The preload library was able to handle this for TCP rsockets
	by using a per thread variable and checking for recursive calls
	coming from rsockets back into the preload library.  The preload
	library would direct such calls to the real socket calls.

	The problem is more complex for UDP rsockets, which can invoke
	socket calls from an internal rsocket thread.  The result is that
	the preload library intercepts socket calls that originate from
	the rsocket library which are not recursive.

	Although, this is really a problem with the preload library,
	the simplest solution is for rsockets to fully initialize the
	library when allocating the first rsocket, versus deferring
	initialization until required.  The preload library can then
	detect the recursive calls.

2012-12-20  Sean Hefty  <sean.hefty@intel.com>

	examples/udpong: Add test program for rsocket datagrams
	Add a sample test program to test datagram rsockets.  Move
	common routines used by udpong and other test programs into
	a common source file.

2012-12-03  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add datagram support
	Add datagram support through the rsocket API.

	Datagram support is handled through an entirely different protocol and
	internal implementation than streaming sockets.  Unlike connected rsockets,
	datagram rsockets are not necessarily bound to a network (IP) address.
	A datagram socket may use any number of network (IP) addresses, including
	those which map to different RDMA devices.  As a result, a single datagram
	rsocket must support using multiple RDMA devices and ports, and a datagram
	rsocket references a single UDP socket, plus zero or more UD QPs.

	Rsockets uses headers inserted before user data sent over UDP sockets to
	resolve remote UD QP numbers.  When a user first attempts to send a datagram
	to a remote address (IP and UDP port), rsockets will take the following steps:

	1. Store the destination address into a lookup table.
	2. Resolve which local network address should be used when sending
	   to the specified destination.
	3. Allocate a UD QP on the RDMA device associated with the local address.
	4. Send the user's datagram to the remote UDP socket.

	A header is inserted before the user's datagram.  The header specifies the
	UD QP number associated with the local network address (IP and UDP port) of
	the send.

	A service thread is used to process messages received on the UDP socket.  This
	thread updates the rsocket lookup tables with the remote QPN and path record
	data.  The service thread forwards data received on the UDP socket to an
	rsocket QP.  After the remote QPN and path records have been resolved, datagram
	communication between two nodes are done over the UD QP.

2012-12-03  Or Gerlitz  <ogerlitz@mellanox.com>

	[librdmacm] Fixed build problem due to missing macro
	rsocket.c wasn't passing compilation as of missing definition for the
	container_of macro, fix it. Reported-by: Eyal Salamon <esalomon@mellanox.com>

2012-11-05  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Remove fscanf build warnings
	Cast fscanf return values to (void) to indicate that we don't
	care if the call fails.  In the case of a failure, we simply
	fall back to using default values.

	Problem reported by Or Gerlitz <ogerlitz@mellanox.com>

2012-10-25  Sean Hefty  <sean.hefty@intel.com>

	riostream: Add example program for using iomap routines.
	riostream is based on rstream, but uses the new riomap, riounmap,
	and riowrite calls instead.  It runs a series of latency and
	bandwidth tests using remote iomapped memory.

	riostream is limited to using zero copy transfers at the
	receiving side only at this time.

2012-10-21  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add APIs for direct data placement
	We introduce rsocket extensions for supporting direct
	data placement (also known as zero copy).  Direct data
	placement avoids data copies into network buffers when
	sending or receiving data.  This patch implements zero
	copies on the receive side, but adds some basic framework for
	supporting it on the sending side.

	Integrating zero copy support into the existing socket APIs
	is difficult to achieve when the sockets are set as
	nonblocking.  Any such implementation is likely to be unusable
	in practice.  The problem stems from the fact that socket
	operations are synchronous in nature.  Support for asynchronous
	operations is limited to connection establishment.

	Therefore we introduce new calls to handle direct data placement.
	The use of the new calls is optional and does not affect the
	use of the existing calls.  An attempt is made to have the new
	routines integrate naturally with the existing APIs.  The new
	functions are: riomap, riounmap, and riowrite.  The basic operation
	can be described as follows:

	1. App A calls riomap to register a data buffer with the local
	   RDMA device.  Riomap returns an off_t offset value that
	   corresponds to the registered data buffer.  The app may
	   select the offset value.
	2. Rsockets will transmit an internal message to the remote
	   peer with information about the registration.  This exchange
	   is hidden from the applications.
	3. App A sends a notification message to app B indicating that
	   the remote iomapped buffer is now available to receive data.
	4. App B calls riowrite to transmit data directly into the
	   riomapped data buffer.
	5. App B sends a notification message to app A indicating that
	   data is available in the mapped buffer.
	6. After all transfers are complete, app A calls riounmap to
	   deregister its data buffer.

	Riomap and riounmap are functionally equivalent to RDMA
	memory registration and deregistration routines.  They are loosely
	based on the mmap and munmap APIs.

	off_t riomap(int socket, void *buf, size_t len,
		     int prot, int flags, off_t offset)

	Riomap registers an application buffer with the RDMA hardware
	associated with an rsocket.  The buffer is registered either for
	local only access (PROT_NONE) or for remote write access (PROT_WRITE).
	When registered for remote access, the buffer is mapped to a given
	offset.  The offset is either provided by the user, or if the user
	selects -1 for the offset, rsockets selects one.  The remote peer may
	access an iomapped buffer directly by specifying the correct offset.
	The mapping is not guaranteed to be available until after the remote
	peer receives a data transfer initiated after riomap has completed.

	int riounmap(int socket, void *buf, size_t len)

	Riounmap removes the mapping between a buffer and an rsocket.

	size_t riowrite(int socket, const void *buf, size_t count,
			off_t offset, int flags)

	Riowrite allows an application to transfer data over an rsocket
	directly into a remotely iomapped buffer.  The remote buffer is specified
	through an offset parameter, which corresponds to a remote iomapped buffer.
	From the sender's perspective, riowrite behaves similar to rwrite.  From
	a receiver's view, riowrite transfers are silently redirected into a pre-
	determined data buffer.  Data is received automatically, and the receiver
	is not informed of the transfer.  However, iowrite data is still considered
	part of the data stream, such that iowrite data will be written before a
	subsequent transfer is received.  A message sent immediately after
	initiating an iowrite may be used to notify the receiver of the iowrite.

	It should be noted that the current implementation primarily focused
	on being functional for evaluation purposes.  Some checks have been
	deferred for subsequent patches, and performance is currently limited
	by linear lookups.

2012-10-17  Roland Dreier  <roland@purestorage.com>

	rdma_xserver/client: Fix man page formatting
	Putting 'r' at the beginning of a line in the nroff source for man pages
	is confusing to nroff because lines that start with a single quote
	character ' or a dot character . are treated as control lines, which is
	not what's intended here.  Some of the man page text ends up left out of
	the formatted output.

	Fix this by just wrapping the text slightly differently in the source
	(which doesn't matter since nroff reflows the text anyway).  Also add a
	missing ".TP" so that the -p and -c options are not run together in the
	formatted output.

2012-10-09  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Disable ACM support if ibacm.port is not found
	The librdmacm will try to connect port 6125 if ibacm.port is
	not found.  The problem is that some other service or application
	could be using that port and respond with garbage.  Rather
	than falling back to a hard coded port number, if ibacm.port
	is not found, simply disable ACM support.

	This has the effect of removing support for older versions
	of ibacm, unless the port file is created manually.

	Patch created based on feedback from Doug Ledford and Florian
	Weimer from RedHat.

2012-10-09  Dotan Barak  <dotanb@dev.mellanox.co.il>

	[5/5,librdmacm] rping: added checks to the return values functions
	This will make rping to exit with return value other than zero in case of an
	error.

	Reviewed-by: Yishai Hadas <yishaih@mellanox.com>

2012-10-09  Dotan Barak  <dotanb@dev.mellanox.co.il>

	[4/5,librdmacm] rstream: added missing return is accept() failed

2012-10-09  Dotan Barak  <dotanb@dev.mellanox.co.il>

	[3/5,librdmacm] rstream: initialize return value in server_connect()
	If use_async == 0 and rs_accept() passes (i.e. non negative value), then
	the return value from the function was uninitialized.

	Reviewed-by: Yishai Hadas <yishaih@mellanox.com>

2012-10-09  Dotan Barak  <dotanb@dev.mellanox.co.il>

	[2/5,librdmacm] rsocket: added missing break
	Reviewed-by: Yishai Hadas <yishaih@mellanox.com>

2012-10-09  Dotan Barak  <dotanb@dev.mellanox.co.il>

	[1/5,librdmacm] rsocket: add missing va_end() after calling va_end()
	Not doing so, may lead to resource leak.

	Reviewed-by: Yishai Hadas <yishaih@mellanox.com>

2012-10-04  Sean Hefty  <sean.hefty@intel.com>

	ucmatose: Remove connect parameter passed into rdma_accept
	Pass in NULL for conn_param into rdma_accept to indicate
	that the passive side will use the values specified by the
	active side.

2012-10-04  Sean Hefty  <sean.hefty@intel.com>

	ucmatose: Fix number of connections to disconnect
	When ucmatose aborts because of issues trying to connect
	to the server, it moves to disconnecting all connections.
	However, not all connections may have been established.
	The result is that ucmatose will hang in disconnect_events.
	Fix this by setting the number of times that we need to
	disconnect to the number of times that we successfully
	connect.

	This problem is based on a report by Doug Ledford
	<dledford@redhat.com>

2012-10-04  Sean Hefty  <sean.hefty@intel.com>

	rping: Reduce retry_count to fit in 3-bits
	retry_count is a 3 bit value on IB, reduce it from
	10 to 7.

	A value of 10 prevents rping from working over the Intel
	IB HCA.  Problem reported by Doug Ledford <dledford@redhat.com>

	The retry_count is also not set when calling rdma_accept.
	Rather than passing different values into rdma_accept than
	what was specified by the remote side, use the values given
	in the connection request.

2012-10-04  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Place container_of inside #ifdef
	verbs.h defines container_of.  Only define it if not defined

	addrinfo: Remove debug printf calls
	These never should have made it into the commit.  :P

2012-09-10  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Document rsocket protocol and design
	Include a brief overview of the rsocket protocol and underlying design
	with the source code to make it easier for someone trying to decipher
	the actual code.

2012-09-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Support using GIDs with rdma_getaddrinfo
	Allow the user to specify a GID as the node parameter into
	rdma_getaddrinfo.

	To distinguish between the node being an IPv6 address or a GID,
	we add a new flag, RAI_FAMILY, which can be set as part of the
	hints to rdma_getaddrinfo.  When set, this flag indicates that the
	value of ai_family in the hints should be used when interpretting
	the node parameter.

2012-09-07  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Fix state checks in dup2
	The patch to add dup2 support was never updated to handle the fd
	state.  The check for the fd type == fd_fork is no longer valid.
	We need to instead check the fd state before handling forking.

	Problem pointed out by Alex Couvrard <acouvrard@gmail.com>

2012-08-29  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Rename ucma_copy_rai_addr to ucma_set_ep_data
	Simple function rename to better indicate operation.

2012-08-28  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Enable AF_IB support

	librdmacm: Set address family for source address returned by ACM
	Set the sa_family type when saving the source address returnd
	by ACM.

2012-08-28  Yann Droneaud  <ydroneaud@opteya.com>

	librdmacm: Report error level in error messages
	Report error messages as either 'Warning' or 'Fatal'.

	librdmacm: Use common prefix for error messages

	librdmacm: Report error messages on stderr

2012-08-23  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Avoid rsocket calls until after fork
	When an rsocket call is made before an application calls fork(),
	the forked applications can hang.  This can be seen by running
	netserver and two netperf clients simultaneously.  The second
	netperf client will eventually stop performing data transfers.

	LD_PRELOAD=librspreload.so netserver -D

	LD_PRELOAD=librspreload.so netperf -v2 -c -C -H 192.168.0.101 -l30
	LD_PRELOAD=librspreload.so netperf -v2 -c -C -H 192.168.0.101 -l30

	It's not clear what the specific problem is.  The best guess is
	that libibverbs or the provider library (e.g. libmlx4) perform
	some initialization, such as mmap'ing device memory, which does not
	work when fork is called.

	As a work-around, avoid calling rsocket routines until immediately
	before they are needed.  This allows the process to fork before
	the libraries are initialized.

2012-08-20  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Fix checks in fork_active/passive
	Fix passing in wrong variable to rconnect(), check state instead
	of type, and move call to getpeername until after we are sure than
	the normal socket connection has completed.

	Problems pointed out by Sridhar Samudrala <sri@us.ibm.com>

2012-08-18  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Re-enable ibacm support
	Commit 272c3cc024d0e5854cbafa6c2f1e8560398a68d7, "Delay ACM
	connection until resolving an address", removed the call to
	ucma_ib_init without adding it back in the correct location.
	As a result, the librdmacm no longer uses ibacm.  Fix this
	by adding the initialization call when resolving an address.

2012-08-17  Sean Hefty  <sean.hefty@intel.com>

	rstream: Use MSG_WAITALL for blocking test

2012-08-16  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Add support for MSG_WAITALL rrecv() flag

2012-08-15  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Add fstat support
	vsftpd calls fstat on a socket.  Fake it out.

2012-08-14  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Support sendfile
	Handle users calling sendfile with an rsocket.

2012-08-14  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Do not block connect when supporting fork
	Many FTP servers require fork support.  However, FTP clients,
	such as ncftp, will perform the following call sequence:

	send PASV request to server over connection 1
	         server will listen for connection 2
	issue nonblocking connect to server
	send ACCEPT request to server over connection 1
	         server will accept connection 2

	The current fork support converts all nonblocking connect
	calls to blocking.  The result is that the FTP client ends up
	blocked waiting for the server to accept the connection,
	which it will never do.

	To handle this case, we have the active side follow the same
	rule as the server side and defer establishing the rsocket
	connection until the user calls the first data transfer routine.

2012-08-14  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Minor cleanup of fork_passive handling
	Minor code cleanup in passive side handling of fork support.

2012-08-08  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Support SO_OOBINLINE
	We don't support urgent data, so just return success.

2012-08-02  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Support dup2 calls
	vsftpd requires dup2() support.  To handle dup2, we need to add
	reference count tracking to the preload fd's.

	rspreload: Call real.close in fd_close
	The index into the preload lookup table is obtained by opening
	/dev/null and use the returned value.  When closing the file,
	use the real close call and not the preload close call.  This
	is a minor optimization, but clarifies the expected operation.

2012-07-30  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Improve disconnect time under normal conditions
	When both sides of a connection attempt to close at the same
	time, one of the two sides can easily get an error when sending
	a disconnect message.  This results in that side hanging
	during close until the send times out.  (The time out is caused
	by the remote side destroying its QP.)

	We can reduce the chance of this occurring by immediately
	assuming that the disconnect has been successful once we've
	received the remote side's disconnect message, or we've
	polled a send completion for the local disconnect message.

2012-07-30  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Use wr_id to determine completion type
	If a work request has completed in error, the completion type
	field is undefined.  Use the wr_id to determine if the failed
	completion was a send or receive.

	This fixes an issue where MPI can hang during finalize.  With
	both sides of a connection shutting down simultaneously, one
	side may complete quicker and delete its QP before the other
	side receives an acknowledgement to their disconnect message.
	Eventually, the disconnect message will time out, but because
	the completion type field is undefined, it may be processed
	as a failed receive, rather than a failed send.  The end
	result is that the second side hangs waiting for the send to
	complete.

	This problem showed up more easily after commit
	2e5b0fc95964f74ea59dd725e849027faa0cd526, but existed beforehand.

2012-07-25  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Enable support for privileged ports
	Allow the preload library to use rsockets with priviledged
	ports.

	rspreload: Call init from getsockname()
	netperf for some unknown reason calls getsockname() using a
	hard coded value of 0, without first allocating a socket.
	This causes the rsocket preload library to crash, since the
	library has not been properly initialized.

2012-07-25  Sean Hefty  <sean.hefty@intel.com>

	rstream: Add option to test fork support
	If the user specifies '-T f', rstream will process
	connections in a child process.  The server continues
	to run until all child processes have completed their
	tests.

	Fork support requires use of the librspreload library.

2012-07-25  Sean Hefty  <sean.hefty@intel.com>

	librspreload: Support server apps that call fork()
	Provide limited support for applications that call fork().  To
	handle fork(), we establish connections using normal sockets.
	The socket is later converted to an rsocket when the user
	makes the first call to a data transfer function (e.g. send,
	recv, read, write, etc.).

	Fork support is indicated by setting the environment variable
	RDMAV_FORK_SAFE = 1.  When set, the preload library will delay
	converting to an rsocket until the user attempts to send or receive
	data on the socket.  To convert from a normal socket to an
	rsocket, the preload library must inject a message on the
	normal socket to synchronize between the client and server.  As
	a result, if the rsocket connection fails, the ability to
	silently fallback to the normal socket may be compromised.  Fork
	support is disabled by default.

	The current implementation works for simple test apps under
	ideal conditions.  Although it supports nonblocking sockets, it
	uses blocking rsockets when migrating connections.

2012-07-19  Sean Hefty  <sean.hefty@intel.com>

	librspreload: Make socket_fallback() call more generic
	socket_fallback is used to switch from an rsocket to a normal
	socket in the case of failures.  Rename the call and make it
	more generic, so that it can switch between an rsocket and
	a normal socket in either direction.  This will be used to
	support fork().

	As part of this change, we move the list of hooked and rsocket
	calls into structures, versus maintaining a large number of
	static variables.

2012-07-19  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Only allocate verbs resources when needed
	The librdmacm allocates a PD per device on initialization.  Although
	we need to maintain the device list while the library is loaded
	(see rdma_get_devices), we can reduce the overhead by only allocating
	verbs resources when they are needed.

	This allows the rsocket preload library to support fork for
	applications that spawn connections off to child processes.

2012-07-19  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Remove unused 'ib' variable from ucma_init

2012-07-12  Sean Hefty  <sean.hefty@intel.com>

	v1.0.16

2012-07-12  Hal Rosenstock  <hal@mellanox.com>

	librspreload: Fix typecast to eliminate compile warnings
	src/preload.c: In function ?bind?:
	src/preload.c:350: warning: assignment from incompatible pointer type
	src/preload.c: In function ?connect?:
	src/preload.c:397: warning: assignment from incompatible pointer type

2012-07-12  Sean Hefty  <sean.hefty@intel.com>

	rspreload: Document use of librspreload library

2012-07-11  sean.hefty@intel.com  <sean.hefty@intel.com>

	librdmacm: Include src/common.h in distribution
	Add missing header file to distribution to allow rpmbuild to
	work.

2012-07-11  Yann Droneaud  <ydroneaud@opteya.com>

	librdmacm: Validate source address protocol family in rdma_resolve_addr
	If a source address is provided but its protocol family is not recognized,
	returns an error.

2012-07-09  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Build librspreload library as part of build
	Build the rsocket preload library as part of the build.  To reduce the
	risk of the preload library intercepting calls without the user's
	knowledge, the preload library is installed into {_libdir}/rsocket.

2012-06-28  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Support IPV6_V6ONLY socket option

2012-06-28  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Handle other shutdown option
	Handle SHUT_RD and SHUT_WR shutdown options.

	In order to handle shutting down the send and receive sides
	separately, we break the connection state into multiple sub-states.
	This allows us to be partially connected (i.e. for either just
	reads or just writes).

	Support for SHUT_WR is needed to handle netperf properly, which
	shuts down a socket by having the client use SHUT_WR, followed by
	the server completing the disconnect with SHUT_RDWR.  The following
	patch eliminates an error message from netperf:

	'shutdown_control: no response received  errno 95'

2012-06-28  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Set readfds event if rsocket has been disconnected

2012-06-11  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add rsocket man page

2012-06-08  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Use configuration files to specify default settings
	Give an administrator control over the default settings
	used by rsockets.  Use files under %sysconfig%/rdma/rsocket as shown:

	mem_default - default size of receive buffer(s)
	wmem_default - default size of send buffer(s)
	sqsize_default - default size of send queue
	rqsize_default - default size of receive queue
	inline_default - default size of inline data

	If configuration files are not available, rsockets will continue to
	use internal defaults.

2012-06-08  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Spin before blocking on an rsocket
	The latency cost of blocking is significant compared to round
	trip ping-pong time.  Spin briefly on rsockets before calling
	into the kernel and blocking.

	The time to spin before blocking is read from an rsocket
	configuration file %sysconfig%/rdma/rsocket/polling_time.  This
	is user adjustable.

	As a completely unintentional side effect, this just happens to
	improve application performance in benchmarks, like netpipe,
	significantly. ;)

2012-06-08  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Handle TCP_MAXSEG socket option
	netperf uses the TCP_MAXSEG socket option.  Add support for it.
	Problem reported by Sridhar Samudrala <sri@us.ibm.com>

2012-06-01  Hal Rosenstock  <hal@mellanox.com>

	man: List RDMA_PS_IB as a supported port space in rdma_getaddrinfo man page

2012-05-29  Sean Hefty  <sean.hefty@intel.com>

	rstream: Use separate connections for latency/bw tests
	Optimize each connection for either latency or bandwidth
	results.  This improves small message latency under 384
	bytes by .5 - 1 us, while increasing bandwidth by
	1 - 1.5 Gbps.

	rstream: Use snprintf in place of sprintf
	Avoid possible buffer overrun.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rstream: Add option to specify size of send/recv buffers

	rsockets: Change the default QP size from 512 to 384
	Simple latency/bandwidth tests using rstream showed minimal
	difference in performance between using a QP sized to 384
	entries versus 512.  Reduce the overhead of a default rsocket
	by using 384 entries.  A user can request a larger size by
	calling rsetsockopt.

	rsockets: Simplify state checks

	rsocket preload: Use environment variable to set QP size
	Allow the user to specify the size of the send/receive queues
	and inline data size through environment variables:
	RS_SQ_SIZE, RS_RQ_SIZE, and RS_INLINE.

	rsocket: Add option to specify size of inline data
	Allow the user to override the default inline data size.
	We still require a minimum size in order to transfer receive
	buffer update message.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Allow user to specify the QP sizes
	Add setsockopt options that allow the user to specify the desired
	size of the underlying QP.  The provided sizes are used as the
	maximum size when creating the QP.  The actual sizes of the QP
	are the smaller of the user provided maximum and the maximum
	sizes supported by the underlying hardware.

	A user may retrieve the actual sizes of the QP through the
	getsockopt call.

	The send and receive queue sizes are specified separately.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Define options specific to rsockets
	Allow a user to control some of the RDMA related attributes
	of an rsocket through setsockopt/getsockopt.  A user specifies
	that the rsocket should be modified through SOL_RDMA level.

	This patch provides the initial framework.  Subsequent patches
	will add the configurable parameters.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Reduce QP size if larger than hardware maximums
	When porting rsockets to iwarp, it was discovered that the default
	QP size (512) was larger than that supported by the hardware.
	Decrease the size of the QP if the default size is larger than
	the maximum supported by the hardware.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rs-preload: Handle recursive socket() calls
	When ACM support is enabled in the librdmacm, it will attempt to
	establish a socket connection to the ACM daemon.  When the rsocket
	preload library is in use, this can result in a recursive call
	to socket() that results in the library hanging.  The resulting
	call stack is:

	socket() -> rsocket() -> rdma_create_id() -> ucma_init() ->
	socket() -> rsocket() -> rdma_create_id() -> ucma_init()

	The second call to ucma_init() hangs because initialization is
	still pending.

	Fix this by checking for recursive calls to socket() in the preload
	library.  When detected, call the real socket() call instead of
	directing the call back into rsockets().  Since rsockets is a part
	of the librdmacm, it can call rsockets directly if it wants to use
	rsockets instead of standard sockets.

	This problem and the cause was reported by Chet Murthy <chet@watson.ibm.com>

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Delay ACM connection until resolving an address
	Avoid creating a connection to the ACM service when
	it's not needed.  For example, if the user of the librdmacm
	is a server application, it will not use ACM services.

	acm: Use -1 to indicate an invalid socket rather than 0
	socket() can return 0 as a valid socket.  This can happen
	when using a daemon that closes stdin/out/err.

2012-05-28  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Fix hang in rrecv/rsend after disconnecting
	If a user calls rrecv() after a blocking rsocket has been disconnected,
	it will hang.  This problem and the cause was reported by Sirdhar Samudrala
	<samudrala@us.ibm.com>.  It can be reproduced by running netserver -f -D
	using the rs-preload library.  A similar issue exists with rsend().

	Fix this by not blocking on a CQ unless we're connected.

2012-05-22  Sean Hefty  <sean.hefty@intel.com>

	rstream: Check for connection error on async connect

2012-05-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Check that send and recv CQs are different before destroying
	ucma_destroy_cqs() destroys both the send and recv CQs if they
	are non-null.  If the two CQs are actually the same one, this
	results in a crash when trying to destroy the second CQ.  Check
	that the CQs are different before destroying the second CQ.

	This fixes a crash when using rsockets, which sets the send and
	recv CQs to the same CQ.

2012-05-19  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Support older acm.h header files
	Older versions of acm.h do not include the resolve_data or
	perf_data fields in struct acm_msg.  If we're using an older
	version of the acm.h header file, use an internal definition
	of struct acm_msg.

2012-05-17  Sean Hefty  <sean.hefty@intel.com>

	rstream: Add test option to include more sizes
	Allow user to specify that a full set of transfer sizes should
	be tested.

	rstream: Group latency/bandwidth tests together
	Rather than grouping tests by transfer size, group by the test type.

2012-05-17  Sean Hefty  <sean.hefty@intel.com>

	rstream: Set rsocket nonblocking if set to async operation
	If asynchronous use is specified (use of poll/select), set the
	rsocket to nonblocking.  This matches the common usage case for
	asynchronous sockets.

	When asynchronous support is enabled, the nonblocking/blocking
	test option determines whether the poll/select call will block,
	or if rstream will spin on the calls.

	This provides more flexibility with how the rsocket is used.
	Specifically, MPI often uses nonblocking sockets, but spins on
	poll/select.  However, many apps will use nonblocking sockets,
	but wait on poll/select.

2012-05-17  Sean Hefty  <sean.hefty@intel.com>

	rstream: Clarify use of async test option

2012-05-17  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rstream: Set rsocket nonblocking for base tests
	The base set of rstream tests want nonblocking rsockets, but don't
	actually set the rsocket to nonblocking.  It instead relies on the
	MSG_DONTWAIT flag.  Make the code match the expected behavior and
	set the rsocket to nonblocking and make nonblocking the default.

	Provide a test option to switch it back to blocking mode.  We keep
	the existing nonblocking test option for compatibility.

2012-05-17  Sean Hefty  <sean.hefty@intel.com>

	rstream: Always set TCP_NODELAY on rsocket
	The NODELAY option is coupled with whether the socket is blocking
	or nonblocking.  Remove this coupling and always set the NODELAY
	option.

	NODELAY currently has no effect on rsockets.

2012-05-10  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rsocket: Succeed setsockopt REUSEADDR on connected sockets
	The RDMA CM fail calls to set REUSEADDR on an rdma_cm_id if
	it is not in the idle state.  As a result, this causes a failure
	in NetPipe when run with socket calls intercepted by rsockets.
	Fix this by returning success when REUSEADDR is set on an rsocket
	that has already been connected.  When running over IB, REUSEADDR
	is not necessary, since the TCP/IP addresses are mapped.

2012-05-08  Sean Hefty  <sean.hefty@intel.com>

	rsockets: Optimize synchronization to improve performance
	Hotspot performance analysis using VTune showed pthread_mutex_unlock()
	as the most significant hotspot when transferring small messages using
	rstream.  To reduce the impact of using pthread mutexes, replace it
	with a custom lock built using an atomic variable and a semaphore.
	When there's no contention for the lock (which is the expected case
	for nonblocking sockets), the synchronization is reduced to
	incrementing and decrementing an atomic variable.

	A test that acquired and released a lock 2 billion times reported that
	the custom lock was roughly 20% faster than using the mutex.
	26.6 seconds versus 33.0 seconds.

	Unfortunately, further analysis showed that using the custom lock
	provided a minimal performance gain on rstream itself, and simply
	moved the hotspot to the custom unlock call.  The hotspot is likely
	a result of some other interaction, rather than caused by slowness
	in releasing a lock.  However, we keep the custom lock based on
	the results of the direct lock tests that were done.

2012-04-24  Dotan Barak  <dotanb@dev.mellanox.co.il>

	rping: Replace sprintf with snprintf to protect from buffer overflow

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Succeed setting SO_KEEPALIVE option
	memcached sets SO_KEEPALIVE, so succeed any requests to set
	that option.  We don't actually implement keepalive at this time.

	To implement keepalive, we would need to record the last time
	that a message was sent or received on an rsocket.  If no
	new messages are processed within the keepalive timeout, then
	we would need to issue a keepalive message.  For rsockets,
	this would simply mean sending a 0-byte control message that
	gets ignored on the remote side.

	The only real difficulty with handlng keepalive is doing it
	without adding threads.  This requires additional work in
	poll to occasionally timeout, send keepalive messages, then
	return to polling if no new data has arrived.  Alternatively,
	we can add a thread to handle sending keepalive messages.

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Succeed SO_LINGER socket option
	Succeed calls to set the SO_LINGER socket option.  We don't
	actually implement SO_LINGER semantics because we never place
	an rsocket into a timewait state.  Unlike socket behavior,
	we do wait for all pending data to be transferred by the hardware.
	This is done so that the disconnect message can be sent over
	the rsocket connection.

	rsocket: Handle socket option toggling on/off
	If the user turns a socket option off, record that, so that
	rgetsockopt returns the correct state of the option.

	rsocket: Discard unrecognized control messages
	If we receive a control message that is not known, simply discard it.
	This provides some ability to support forward compatibility.

	rsocket: Work-arounds to support RH EL5
	Discard ENOSYS errors when trying to set address reuse.

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Allow use of LD_PRELOAD to intercept socket calls
	Intercept socket calls and convert TCP socket operation to
	streaming over RDMA.

	Allow falling back from rsockets to normal sockets on error
	or when trying to bind/connect to a reserved port.  This is
	needed to handle MPI job startup, where MPI should use rsockets,
	but mpiexect needs to communicate using ssh over normal sockets.

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add sample application to copy files over rsockets
	rcopy will copy files from a source system to a specified remote
	server.  It's essentially a really dumb FTP type program that can
	be used to quickly transfer files between systems, which can be
	useful to verify data integrity.

	(It was easier to create this program than modify an existing FTP
	client and server application, which was my first choice.  Fork
	support is difficult.)

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	rsocket: Add example program that uses rsocket
	rstream provides an example that uses either rsocket or socket
	APIs.  The latter allows rstream to be used to verify rsocket
	behavior compared to socket.

2012-04-16  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Define streaming over RDMA interface (rsockets)
	Introduces a new set of APIs that support a byte streaming interface
	over RDMA devices.  The new interface matches sockets, except that all
	function calls are prefixed with an 'r'.

	The following functions are defined:

	rsocket
	rbind, rlisten, raccept, rconnect
	rshutdown, rclose
	rrecv, rrecvfrom, rrecvmsg, rread, rreadv
	rsend, rsendto, rsendmsg, rwrite, rwritev
	rpoll, rselect
	rgetpeername, rgetsockname
	rsetsockopt, rgetsockopt, rfcntl

	Functions take the same parameters as that use for sockets.  The
	follow capabilities and flags are supported at this time:

	PF_INET, PF_INET6, SOCK_STREAM, IPPROTO_TCP
	MSG_DONTWAIT, MSG_PEEK
	SO_REUSEADDR, TCP_NODELAY, SO_ERROR, SO_SNDBUF, SO_RCVBUF
	O_NONBLOCK

	The rpoll call supports polling both rsockets and normal fd's.

2012-04-04  Sagi Grimberg  <sagig@mellanox.com>

	ucmatose: Fix segfault on address error
	Client connect_events() shoudl fail if it received some error,
	otherwise the program will try to reach a non-existent QP
	resource resulting in a segfault.  Return an error from
	cma_handler() if we had a connection error.

2012-03-02  Sean Hefty  <sean.hefty@intel.com>

	Automatically detect if ibacm is installed
	If the ibacm header file is available, automatically have the
	librdmacm configured to use it.  This removes the --with-ib_acm
	configure option.

	Update rdma_disconnect to indicate both sides should call it.
	rdma_disconnect should be called from both sides to quickly disconnect.
	Clarify this in the man page.

	librdmamcm: Check for valid route in ucma_set_ib_route
	ucma_set_ib_route will call rdma_getaddrinfo to obtain IB path
	information.  However, rdma_getaddrinfo will return success,
	but not provide routing data if no route can be found (the IB
	ACM service is not running).  In this case, we can call
	rdma_set_option without a valid route.  Although the kernel
	will trap this and fail, we can detect the error in the library.
	This will speed up the connection rate if IB ACM is not in use.

	librdmacm: Fix warning 'resolve_msg' breaks aliasing rules

2012-03-02  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Return an error if user specifies AF_IB but it is not supported
	If the user specifies an AF_IB address into rdma_bind_addr,
	rdma_resolve_addr, rdma_join_multicast, or rdma_leave_multicast,
	but the kernel does not support AF_IB return an error.

	Note that rdma_getaddrinfo will never return an AF_IB address to the
	user unless kernel support is present.  A application would need
	to construct and AF_IB address by hand before making one of the
	above mentioned calls.  This check prevents overrunning the
	command buffer written to the kernel.

2012-03-02  Sean Hefty  <sean.hefty@intel.com>

	udaddy: Update udaddy to use rdma_getaddrinfo

	cmatose: Replace use of getaddrinfo with rdma_getaddrinfo
	Now that rdma_getaddrinfo exists, use it rather than getaddrinfo.
	This will eventually allow us to specify native IB addresses into
	cmatose once AF_IB support is there.

	librdmacm: Report AF_IB as second rdma_addrinfo
	If AF_IB is supported, the librdmacm will attempt to convert
	AF_INET or AF_INET6 to AF_IB.  Rather than replacing the
	AF_INET/6 rdma_addrinfo, provide the AF_IB addresses as a
	second rdma_addrinfo linked from the AF_INET/6 version.

	librdmacm: Set errno correctly in ucma_complete
	The status value is negative, convert it to positive before setting errno.

	rdma_verbs: Set errno correctly in rdma_get_send/recv_comp

2011-12-15  Sean Hefty  <sean.hefty@intel.com>

	Merge branch 'sor'

	librdmacm: Update web site and email addresses

2011-12-12  Sean Hefty  <sean.hefty@intel.com>

	Merge branch 'sor'

2011-12-12  Or Gerlitz  <ogerlitz@mellanox.com>

	udaddy/ucmatose: allow easy setting of tos in hex
	Under IBoE, the 3 MSBits of the TOS map to the SL, hence letting
	the user to specify them in hex makes the interface friendlier.

2011-12-08  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Return ECONNREFUSED from rdma_connect on reject
	Make the errno return code from rdma_connect constistent with
	connect.  The underlying status value is available by reading
	the event data.

2011-10-31  Dotan Barak  <dotanb@dev.mellanox.co.il>

	rdma/cma: minor code refactoring when saving a string content
	In this case, using strdup will provide a cleaner code
	(and maybe a little bit faster too).

2011-10-26  Dotan Barak  <dotanb@dev.mellanox.co.il>

	librdmacm/udaddy: Fix resource leak in case of error

2011-09-29  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Verify size of route_len
	If the user specifies route information on input to rdma_getaddrinfo,
	verify that the size of the routing data is something that we're
	prepared to handle.

	The routing data is only useful if IB ACM is enabled and may be
	either struct ibv_path_record or struct ibv_path_data on input.

2011-09-27  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Fix duplicate free of connect
	The connect data stored with the cma_id_private is freed in
	rdma_connect, since it is no longer needed.  Avoid duplicating
	the free in rdma_destroy_id by checking for connect_len = 0,
	rather than connect to be NULL.

2011-09-27  Sean Hefty  <sean.hefty@intel.com>

	rdma/verbs: Fix race polling for completions
	To avoid hanging in rdma_get_send/recv_comp, we need to rearm
	the CQ inside of the while loop.  If the CQ is armed,
	the HCA will write an entry to the CQ, then generate a CQ
	event.  However, a caller could poll the CQ, find the entry,
	then attempt to rearm the CQ before the HCA generates the CQ
	event.  In this case, the rearm call (ibv_req_notify_cq) will
	act as a no-op, since the HCA hasn't finished generating the
	event for the previous completion.  At this point, the event
	will be queued.

	A call to ibv_get_cq_event will find the event, but not
	a CQ entry.  The CQ is now not armed, and a call to
	ibv_get_cq_event will block waiting for an event that will
	never occur.

	Problem was found in an rdma_cm example test under development.
	The test can ping-pong messages between two applications.

2011-09-14  Sean Hefty  <sean.hefty@intel.com>

	v1.0.15

2011-08-24  Dotan Barak  <dotanb@dev.mellanox.co.il>

	librdmacm: Fix resource in rdma_migrate_id() error flow
	Prevent resource leak by destroying the event channel before returning from
	function in an error flow.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Fix resource leak when CMA_CREATE_MSG_CMD_RESP fails
	If resources are allocated before CMA_CREATE_MSG_CMD_RESP or
	CMA_CREATE_MSG_CMD are called, and those calls fail, we need
	to cleanup the resources before returning.

	Fix this by changing the CMA_CREATE_MSG macros to remove the
	alloca and calling return.  The request and response structures
	are now declared directly on the stack.  To accomplish this,
	we merge the abi header definition into each command structure.

	Problem reported by: Dotan Barak <dotanb@dev.mellanox.co.il>

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	rdma_xserver/client: Add new test apps
	Add new versions of the rdma_server and rdma_client tests that
	support other types of connections and show how to use more
	RDMA features.  We keep the existing rdma_server and rdma_client
	tests as simple examples.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Do not wait in rdma_accept for UD QPs
	There are no additional connection events to process for UD QPs
	after calling rdma_accept().  When using synchronous rdma_cm_id's,
	simply return to the user after sending the reply.  Do not wait
	for additional events.

	This fixes a hang on the server side when setting up UD QP
	communication.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Specify QP type separately from port space
	We need to know the QP type separately from the port space.  In
	order to support XRC, UC, and other QP types, we use RDMA_PS_IB,
	which no longer provides a 1:1 mapping between the port space
	and QP type.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Abstract ibverbs SRQ creation
	Support QPs with SRQs.  If a user allocates an SRQ on an
	rdma_cm_id, we post receive messages directly to the SRQ.
	This also allows us to handle XRC SRQs, which may be associated
	with an rdma_cm_id, but without a corresponding QP.

	To handle registering memory, we store the PD associated
	with an rdma_cm_id directly with the id, rather than finding
	the PD using a QP pointer.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Add support for XRC qp types
	Support XRC send/receive qp types.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Renumber RDMA_PS_IB to match kernel patch
	RDMA_PS_IB is only a placeholder and not usable yet.  Update
	the assigned value to match that specified for the kernel.

	Update rdma_getaddrinfo to use the port space when formatting
	responses.

2011-08-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Limit autotools output

2011-08-22  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Use union with sockaddr structures
	To avoid strict aliasing compiler warnings, use an unamed union
	to store the src and dst addresses.  This eliminates the need
	for padding and sockaddr casts.

	librdmacm: Fix crash in rdma_connect
	When using rdma_connect for UD QP lookup, there may not be any
	QP associated with the rdma_cm_id.  Plus there may not be any use
	for the conn_param parameter.  Allow conn_param to be optional
	in this situation.  This fixes a crash exposed by rdma_xclient
	sample using XRC QPs.

2011-05-27  Yann Droneaud  <ydroneaud@opteya.com>

	librdmacm: Fail ucma_init if ibv_get_device_list is empty
	From the ibv_get_device_list man page:

	   ibv_get_device_list() returns the array of available RDMA devices, or
	   sets errno and returns NULL if the request fails. If no devices are
	   found then num_devices is set to 0, and non-NULL is returned.

	The librdmacm handles the failure case, but not the case where no
	devices are found.  Handle that case as well.

2011-05-10  Sean Hefty  <sean.hefty@intel.com>

	rdma_server: fix typo in print

2011-04-15  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: define REUSEADDR option
	Support equivalent of SO_REUSEADDR socket option.  When specified
	the rdma_cm_id will be bound to a reuseable address.  This will
	allow other users to bind to that same address.  This is needed
	to support lustre on clusters larger than 1024 nodes.

2011-04-01  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: Size of private data for accept is 196, not 160
	The rdma cm header is not used on the reply.

2011-03-29  Or Gerlitz  <ogerlitz@mellanox.com>

	Document the fact that errno of EISCONN (Transport endpoint is already connected) isn't a failure

2011-03-28  Or Gerlitz  <ogerlitz@mellanox.com>

	librdmacm/doc: Document private data length limitations
	Document the limitations on the user provided private data length
	over Infiniband networks. These limitations are calculated by
	subtracting the rdma-cm header size (see IBA Annex A11 "RDMA CM IP
	Service") from IB's private data len for the REQ (rdma_connect) and
	REP (rdma_accept) messages

2011-03-14  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: fixup rdma_accept documentation for responder_resources
	Responder_resources may be greater than that specified in the
	connect request, up to the maximum supported by the device.

2011-03-10  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Document to set accept_local sysctl for loopback
	For loopback connections between ports on a single system to work,
	accept_local sysctl must be set to 1.  Add this to the readme.

2011-03-05  Sean Hefty  <sean.hefty@intel.com>

	Only allow a user to allocate a single QP off an rdma_cm_id
	Add a simple check to rdma_create_qp() to see if a QP has already been
	associated with an rdma_cm_id.  Otherwise a user can allocate a
	second QP with an ID, with the reference to the first QP replaced
	by the second allocation.

2011-03-04  Sean Hefty  <sean.hefty@intel.com>

	Update rdma_join_multicast documentation to include attaching QP
	The man pages for rdma_join_multicast are not clear on how the
	user can attach the QP to the multicast group.  Clarify this
	behavior.

2010-12-15  Roland Dreier  <rolandd@cisco.com>

	librdmacm: Add a check for libpthread during librdmacm configure.
	Add a check for libpthread during librdmacm configure.  This will add
	libpthread to the list of libraries that librdmacm is linked to.
	Currently librdmacm gets libpthread implicitly through libibverbs, but
	this breaks when using a linker that does not implicitly link with such
	dependencies; eg the new gold linker is such a linker:

	<http://wiki.debian.org/qa.debian.org/FTBFS#A2009-11-
	02Packagesfailingbecausebinutils-gold.2BAC8-indirectlinking>

	Addresses: http://bugs.debian.org/555380
	Addresses: https://bugs.launchpad.net/ubuntu/+source/librdmacm/+bug/687983

2010-12-15  Sean Hefty  <sean.hefty@intel.com>

2010-12-09  Sean Hefty  <sean.hefty@intel.com>

	v1.0.14

2010-12-09  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Support RAI_NUMERICHOST and no delay options
	Add support similar to getaddrinfo AI_NUMERICHOST.  This
	indicates that lengthy address resolution protocols should
	not be used.  Also allow a caller of rdma_getaddrinfo to
	indicate that lengthy route resolution protocols should not
	be used.

	Since rdma_getaddrinfo is a synchronous call, this allows a
	user to obtain locally available data only without long
	delays that may block an application thread.  Callers can then
	use the asynchronous librdmacm calls to complete any missing
	information.

2010-12-02  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: support non-default ACM port number
	By default, ACM uses port 6125.  The actual port number
	used is now published in /var/run/ibacm.port.  Attempt to
	obtain the correct port number from here, and if that fails
	revert to using the default port number of 6125.

2010-11-01  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rping: Make sure CQ event thread exits before destroying the CQ
	It is possible for the CQ event thread to poll the CQ after it has been
	destroyed which can result in a seg fault on T3 interfaces.  This patch
	waits for the thread to exit before destroying the CQ.

2010-11-01  Jonathan Rosser  <jrosser@rd.bbc.co.uk>

	librdmacm: fix compiler warning of void * arithmetic
	void * pointer arithmetic is non-standard.

2010-11-01  Jonathan Rosser  <jrosser@rd.bbc.co.uk>

	librdmacm: fix make install
	make install fails if the include files in the install prefix
	include/rdma,infiniband already exist. install claims that the <src>
	and <destination> file are the same and exits with an error.

	This patch modifies Makefile.am so that the rdma and infiniband include
	files explicitly reference the source directory rather than the build
	directory.

	Also, EXTRA_DIST now only lists files that are not referenced anywhere
	else in Makefile.am

2010-11-01  Jason Gunthorpe  <jgunthorpe@obsidianresearch.com>

	librdmacm: Fix autotools to include the necessary M4 files
	Otherwise running autogen.sh with a new version of autotools and then
	building on a system with an older version tends to explode.
	Unfortunately this is sometimes necessary since the new version is
	required by the package.

	This is how GNU envisions this mess works at least..

2010-10-20  Steve Wise  <swise@opengridcomputing.com>

	RPING: Remove printf for FLUSH completion.

2010-10-18  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: do not modify qp_init_attr in rdma_get_request
	rdma_create_qp modifies the qp_init_attr structure passed in
	by the user to return the actual QP capabilities that were
	allocated.  If the qp_init_attr does not specify CQs, the
	librdmacm will allocate CQs for the user and return them.

	rdma_get_request will allocate a QP off newly connected rdma_cm_id
	if the corresponding listen request is associated with a
	qp_init_attr structure.  The librdmacm passes in the listen->
	qp_init_attr structure into the rdma_create_qp call.
	rdma_create_qp ends up modifying the qp_init_attr's associated
	with the listen.  The result is that future calls to
	rdma_get_request will use the modified qp attributes, rather
	than those specified by the user.

	Fix this by having rdma_get_request pass in a copy of the
	qp_init_attr, rather than modifying those associated with the
	listen.  Also update the man page for rdma_create_qp to indicate
	that the qp_init_attr structure may be modified on output.

2010-10-05  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: only allocate qp in rdma_create_ep if qp_attr provided
	The comments and documentation for rdma_create_ep indicate that
	it will only allocate a QP if initial QP attributes are provided.
	However, the code always attempts to create a QP off an associated
	active rdma_cm_id endpoint.

	By _not_ allocating the QP, this allows a user to first determine
	what RDMA device a rdma_cm_id was associated with.  The user can
	then create a QP that references an existing CQ, SRQ, or PD.

2010-08-20  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: expand support for hints to rdma_getaddrinfo
	If a user passes in hints into rdma_getaddrinfo, they can
	specify resolved source and destination addresses.  In this
	case, there's no need for the user to specify the node or
	service parameters.  This differs from getaddrinfo, which
	indicates that either node or service must be provided, but
	is useful if rdma_getaddrinfo is being used to obtain
	routing data.

	Supporting this option allows the librdmacm to call
	rdma_getaddrinfo internally from rdma_resolve_route when IB ACM
	is enabled.

	In addition to specifying the source and destination addresses
	as part of the hints, a user could instead specify partial
	routing data and rdma_getaddrinfo can resolve the full route.

	This helps to support MPI applications that exchange endpoint
	data, such as LIDs, out of band, but require SL data from the
	SA to avoid potential deadlock conditions.

2010-08-20  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: do not pass uninitialized ai_hints into getaddrinfo
	If rdma_getaddrinfo is called with hints set to NULL, then an
	uninitialized ai_hints structure will be passed into getaddrinfo.

2010-08-10  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.13

	librdmacm: document event status field
	Clarify the value returned in the event status field.

	librdmacm/man: fix typos in man pages

	librdmacm: specify return value in man pages
	Document the return value of all calls in their respective man pages.

	librdmacm: fix all calls to set errno
	The librdmacm documentation (rdma_cm.7 man page) specifies that librdmacm
	functions return 0 on success and -1 on error, with errno set correctly.
	Update places in the code where errno is not set correctly and
	cleanup setting the error code.

2010-07-23  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: convert ibv error values to errno
	The librdmacm documentation (rdma_cm.7 man page) specifies that librdmacm
	functions return 0 on success and -1 on error, with errno set correctly.
	The libibverbs abstractions simply pass the libibverbs return codes
	through to the user.  Since libibverbs may return errno directly through
	a given call, convert the return status to use errno where appropriate.

	This fixes an issue with rdma_get_send_comp and rdma_get_recv_comp, where
	a return value of 1 could indicate both success (1 completed request
	returned) and failure (EPERM error).

2010-07-19  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: add man pages for calls in rdma_verbs.h
	rdma_verbs.h define several inline functions that wrap around verb
	routines.  Add man pages for these calls.

	librdmacm/man: add rdma_get_request man page to release

2010-06-01  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: remove 32-bit build warnings

	librdmacm: support 2.6.9
	Redhat 4.x is based on 2.6.9.  Add support for older kernels.

2010-05-24  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.12

	librdmacm: fix makefile installation path

2010-05-22  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: add man pages for new APIs

2010-05-17  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: define struct ibv_path_record if not defined
	librdmacm relies on struct ibv_path_record from libibverbs.  However,
	to support older versions of libibverbs, define struct ibv_path_record
	if it is not already defined.

2010-05-11  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: disable AF_IB support
	To avoid potential compatability issues, disable AF_IB support
	until it has been queued for inclusion upstream.  We will re-enable
	AF_IB support after releasing version 1.0.12 for OFED 1.5.2, which
	will include support for IB ACM.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: initialize path_cnt

	librdmacm: use IB ACM to resolve IB path
	Starting with 2.6.33, the kernel supports the ability to
	manually specify the path record that a connection should
	use.  Allow the librdmacm to contact the IB ACM to acquire
	path record data, even if rdma_getaddrinfo is not used and
	the kernel does not support AF_IB.

	librdmacm: check if kernel supports AF_IB
	Add check during initialization to determine if the kernel
	supports AF_IB.

	librdmacm/mckey: use AF_IB for unmapped multicast addresses
	If the user joins an unmapped multicast address, use AF_IB,
	rather than AF_INET6, to communicate that information with the
	kernel.

	librdmacm: update man pages
	Update man pages to reflect recent changes to the APIs.

	librdmacm/rdma_server: add new sample server application
	Provide a simple server application to demonstrate the minimal
	amount of coding needed to accept a connection request from
	a client and exchange messages.

	librdmacm/rdma_client: add new client sample
	Provide a very simple client application that shows the
	minimal coding needed to establish a connection and exchange
	messages with a server.  The client makes use of the new
	rdma_getaddrinfo and rdma_create_ep calls, plus rdma verbs
	abstractions.

	librdmacm: provide abstracted verb calls
	Provide abstractions to the verb calls to simplify the user
	interface for more casual verbs consumers.  Users still have
	access to the full range of verbs functionality by calling
	verbs directly.

	librdmacm: format IB CM private data RDMA CM header
	When IB ACM is used, the address and route resolution is
	done entirely in user space.  Before converting AF_INET or
	AF_INET6 addresses to AF_IB, format the connection private
	data for IB CM REQ messages.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: use sockaddr_ib addressing if IB ACM is in use
	If IB ACM route resolution succeeds, provide the user with
	AF_IB addresses, rather than AF_INET or AF_INET6 addressing.
	AF_IB identifies the local and remote devices directly,
	eliminating the need to perform address resolution a second
	time via rdma_resolve_addr.

	AF_IB addresses are returned using sockaddr_ib as part of
	rdma_getaddrinfo.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: define RDMA_PS_IB
	AF_IB uses the IB port space.

	librdmacm: add support for IB ACM service
	Allow the librdmacm to contact a service via sockets to obtain
	address mapping and path record data.  The use of the service
	is controlled through a build option (with-ib_acm).  If the
	library fails to contact the service, it falls back to using
	the kernel services to resolve address and routing data.

	librdmacm: set src_addr in rdma_getaddrinfo
	If the user provides a source address as a hint, copy
	the address to rdma_addrinfo ai_src_addr.

	librdmacm: auto create QPs in rdma_get_request
	If the user passed initial QP attributes into rdma_create_ep,
	allocate a QP for the user as part of rdma_get_request.

	librdmacm: add support for passive side in rdma_create_ep
	Allow calling rdma_create_ep for a listening rdma_cm_id.

	librdmacm: add rdma_destoy_ep
	Provide counterpart call to rdma_create_ep.

	librdmacm: add new call to create id
	Provide a simple call to create an rdma_id, with optional QP.  The
	id is created using synchronous operation, using the output of
	rdma_getaddrinfo.

	librdmacm: specify qp_type when creating id
	To support AF_IB / PS_IB, we need to specify the qp type when
	creating the rdma_cm_id.  The kernel requires this in order
	to select the correct type of operation to perform (e.g. SIDR
	versus REQ).

	librdmacm: define options to set IB route directly
	Add the definitions needed to set IB path record data using set_option.
	This will be used by rdma_create_ep().

	librdmacm: add rdma_get_request
	To simplify passive side operation and better support synchronous
	operations, add rdma_get_request().  This function is called on the
	listening side to retrieve a connection request event.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: add rdma_getaddrinfo
	Provide a call similar to getaddrinfo for RDMA devices and
	connections.  rdma_get_addrinfo is modeled after getaddrinfo, with
	the following modifications:

	A source address is returned as part of the call to allow the
	user to allocate the necessary resources for connections.

	Optional routing information may be returned to support
	Infiniband fabrics.  IB routing information includes necessary
	path record data.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: expose ucma_init to other internal modules
	Remove static property from ucma_init and expose its
	definition in cma.h.  The address resolution module will
	need access to this function.

	librdmacm: allow conn_param to be optional
	If conn_param is not provided to rdma_connect or rdma_accept,
	then default values are used to establish the connection.

	librdmacm: allow user to specify max RDMA resouces
	Allow the user to indicate that the library should select the
	maximum RDMA read values available should be used when
	establishing a connection.  The library selects the maximum
	based on local hardware limitations and connection request
	data.

	librdmacm: make CQs optional for rdma_create_qp
	Allow the user to specify NULL for the send and receive CQs when
	creating a QP through rdma_create_qp.  The librdmacm will automatically
	create CQs for the user, along with completion channel.

	librdmacm: allow pd parameter to be optional
	Allow the user to create a QP using rdma_create_qp without
	specifying a PD.  If a PD is not given, a default PD will be
	used instead.  This simplifies the user interface.

	librdmacm: support synchronous rdma_cm_id's
	Allow the user to specify NULL as the rdma_event_channel in
	order to indicate that the rdma_cm_id should process all requests
	synchronously.

	librdmacm: use calloc call
	Use calloc to allocate and clear memory.

	librdmacm: move common definitions to internal header file

	librdmacm: add support for AF_IB multicast addresses
	Allow the user to specify AF_IB addresses when joining
	multicast groups.  AF_IB addresses contain MGIDs directly.

	librdmacm: add support for AF_IB to resolve_addr
	Allow user to specify AF_IB addresses to rdma_resolve_addr.

	librdmacm: add support to bind using AF_IB
	Allow rdma_bind_addr to accept AF_IB addresses.

	librdmacm: add support for sockaddr_ib length
	Enhance ucma_addrlen to return the correct address size
	for AF_IB.

	librdmacm: replace query_route call with separate queries
	To support other address families and multiple path records,
	replace the query_route call with specific query calls to obtain
	only the desired information.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: add support to query GIDs
	Support query GID ABI to obtain GID information separately from
	path record data and sa_family addressing.

	This patch also adds the definition for sockaddr_ib for user space.

2010-05-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: add ability to query IB path records
	The current query_route command only supports 2 path records.
	Add support for query_path, which is capable of supporting
	multiple paths.

2010-04-07  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: support querying AF_IB addresses
	The current query route command returns path record data and address
	information.  The latter is restricted to sizeof(sockaddr_in6).  In
	order to support AF_IB, modify the library to use the new query addr
	command, which supports larger address sizes and avoids querying for
	path records data when none are available.

	librdmacm: name changes to indicate only IP addresses supported
	Several commands to the kernel RDMA CM only support IP addresses
	because of limitations in the structure definition.  Update
	the library to match the name changes in the kernel and indicate
	that only IP addresses can be used with the current commands.

2010-03-22  Or Gerlitz  <ogerlitz@voltaire.com>

	Subject: [PATCH] librdmacm: document/clarify the delivery of connection established event
	Applications based on the rdma-cm may assume that established event
	is always delivered by the the kernel stack, clarify that.

2010-02-05  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: transition QP to RTS before sending reply
	In order to handle a race condition where the passive side of
	a connection can receive data on a QP before the connection established
	event has been received, transition the QP to RTS before sending the reply.
	This allows a user to send a response to any received message immediately,
	rather than waiting until the connection established event has been
	processed.

	A similar fix was applied to the kernel rdma_cm a while ago.
	Simply duplicate the fix in the user space library.

2009-11-24  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.11

2009-11-14  Or Gerlitz  <ogerlitz@voltaire.com>

	librdmacm/mckey: report multicast events to the user
	Extend mckey to report multicast events.  If mckey joins as a server,
	it can be used to report membership join or ip address changes to
	the  user.

2009-11-13  David J. Wilder  <[dwilder@us.ibm.com]>

	librdmacm/cmatose: add support for ipv6

2009-11-09  Or Gerlitz  <ogerlitz@voltaire.com>

	librdmacm/mckey: enforce local binding for unmapped multicast address
	Enforce that mckey is bound to a local address when specifying
	an unmapped multicast address.  Otherwise, mckey crashes when
	attempting to use cmd_id->verbs pointer.

	Update documentation on using unmapped MGIDs.

2009-10-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.10

	librdmacm/examples: print errno on failures
	Replace printf(... ret) with perror to display the errno value now
	that the library sets errno consistently.

2009-10-21  Jason Gunthorpe  <jgunthorpe@obsidianresearch.com>

	librdmacm: returns errors from the library consistently
	Remove the return of -errno and always return codes via errno.
	As documented in librdmacm, these libraries are already documented
	to return -1 to indicate the code is in errno.

	Update rping to show correct error reporting methodology.

	Also fix errant return of 0 if the read/write syscalls return 0.

2009-10-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: initialize correct pthread condition in rdma_join_multicast
	rdma_join_multicast re-initializes id_priv->cond rather than mc->cond.
	Fix this.  Bug reported by Nir Naaman.

2009-10-19  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: fix race initializing library
	Multi-threaded code can race with ucma_init().  When ucma_init() is called
	it can set cma_dev_cnt before it is done initializing the library.  If
	a second thread checks cma_dev_cnt and finds it non-zero, then it will
	skip the call to ucma_init() and assume that the library is ready for use.
	This can lead to an application crash during startup.

	Do not set cma_dev_cnt until the end of ucma_init(), after all
	initialization has completed.  Adjust the error handling code in
	ucma_init() accordingly.

	Bug reported by Nir Naaman.

2009-09-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.9

2009-01-14  Sean Hefty  <sean.hefty@intel.com>

	cmatose: avoid missing completions
	cmatose uses a single CQ for send and receive completions.  It then
	counts completions to determine if all sends and receives are done.
	It's possible for a receive completion to be polled when the intent
	is to count send completions.  (See server side polling for sends
	done, then receives done.  The poll will get up to 8 completions,
	which can lead to sends and receives being polled together.)

	Fix this by separating the send and receive completions to their
	own CQs to avoid any issues knowing what type of receives have been
	polled from the CQ.

2008-08-14  Aleksey Senin  <alekseys@voltaire.com>

	rping: add ipv6 support

2008-08-11  Aleksey Senin  <alekseys@voltaire.com>

	librdmacm: use sockaddr_storage
	Replace padding and use sockaddr_storage instead.

2008-07-03  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.8

	librdmacm: report timewait exit event

2008-07-02  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rping: allow specifying hostnames in place of IP addresses
	Problem reported by Jeff Squyres <jsquyres@cisco.com>

	librdmacm/rping: fix duplicate usage message
	Avoid displaying the usage message twice if --help option is used.
	Problem reported by Jeff Squyres <jsquyres@cisco.com>.

2008-07-02  Or Gerlitz  <ogerlitz@voltaire.com>

	librdmacm: implement address change event
	RDMA_CM_EVENT_ADDR_CHANGE event can be used by librdmacm consumers
	that wish to have their RDMA sessions always use the same links
	(eg <hca/port>) as the IP stack does. In the current code, this
	does not happen when bonding is used and fail-over happened,
	but the IB link used by an already existing session is operating fine.

	The kernel rdma-cm code was enhanced to use netevent notification
	for sensing that a change has happened in the IP stack, and deliver
	this event for ID that is misaligned that respect with the IP
	stack. The user can act on the event or just ignore it

2008-06-10  Aaron Fabbri  <aafabbri@cisco.com>

	Make manpage reflect the fact that rdma_bind_addr(port=0) does not return the assigned port.

2008-05-22  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: fix license text
	The license text for references a third software license
	that was inadvertently copied in.  Update the license to match that used
	by openfabrics.  This update was based on a request from HP.

2008-04-08  Dotan Barak  <dotanb@dev.mellanox.co.il>

	librdmacm/examples: add option to manually set port number

	librdmacm/examples
	Fix some typos.

2008-03-28  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.7

2008-02-20  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: add rdma_migrate_id
	This is based on user feedback from Doug Ledford at RedHat:

	Events that occur on an rdma_cm_id are reported to userspace through
	an event channel.  Connection request events are reported
	on the event channel associated with the listen.  When the
	connection is accepted, a new rdma_cm_id is created and automatically
	uses the listen event channel.  This is suboptimal where the user
	only wants listen events on that channel.

	Additionally, it may be desirable to have events related to
	connection establishment use a different event channel than those
	related to already established connections.

	Allow the user to migrate an rdma_cm_id between event channels.

2008-02-20  Roland Dreier  <rdreier@cisco.com>

	Add BuildRequires for libibverbs 1.1 to RPM spec file
	librdmacm can't build without libibverbs 1.1.

	Move man pages from man1 into -utils package
	Move the man pages for the executables that are shipped with the -utils
	package from the -devel package into the -utils package itself.

2008-02-20  Roland Dreier  <rolandd@cisco.com>

	Add changelog to rpm spec file
	rpmlint says:

	    E: no-changelogname-tag

2008-02-20  Roland Dreier  <rolandd@cisco.com>

	Remove period from end of Summary in RPM spec file
	rpmlint says:

	    W: summary-ended-with-dot Userspace RDMA Connection Manager.

2008-02-20  Roland Dreier  <rolandd@cisco.com>

	Take empty ChangeLog out of RPM packages
	rpmlint says:

	    E: zero-length /usr/share/doc/librdmacm-1.0.6/ChangeLog

	and there's really no point in shipping an empty file, so just don't
	package it.

2008-02-20  Roland Dreier  <rolandd@cisco.com>

	Fix Source URL in RPM spec file
	Source packages are now in downloads/rdmacm/, not just downloads/.

2008-02-12  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/cm: set status correctly for rejects
	The status is not being set for rejected connection requests.
	Set it by moving common initialization outside of the switch.

2008-01-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.6

2008-01-21  Roland Dreier  <rolandd@cisco.com>

	Update %install secion of librdmacm spec file
	Change from using the %makeinstall macro to using "make install"
	directly.  The page <http://fedoraproject.org/wiki/Packaging/Guidelines>
	has this to say:

	  "Fedora's RPM includes a %makeinstall macro but it must NOT be used
	   when make install DESTDIR=%{buildroot} works. %makeinstall is a
	   kludge....
	   It is error-prone and can have unexpected effects....
	   It can trigger unnecessary and wrong rebuilds....
	   ....it can cause broken *.la files to be installed....
	   Instead, Fedora packages should use: make DESTDIR=%{buildroot}
	   install or make DESTDIR=$RPM_BUILD_ROOT install"

	The librdmacm package uses automake, which means that the "make
	DESTDIR=... install" method works fine, so we should use it.

2008-01-21  Roland Dreier  <rolandd@cisco.com>

	Updated License: field in librdmacm spec file
	Update License: field to match the exact format given in
	http://fedoraproject.org/wiki/Packaging/LicensingGuidelines
	for a package available under a choice of GPL or BSD license.

2008-01-04  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/cm: override default responder_resources with user value
	By default, the responder_resources parameter is set to that received
	in a connection request.  The passive side may override this value
	when accepting the connection.  Use the value provided by the passive
	side when transitioning the QP to RTR state, rather than the value
	given in the connect request.  Without this change, the RTR transition
	may fail if the passive side supports fewer responder_resources than
	that in the request.

2008-01-02  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.5

	librdmacm/man: more updates to man pages
	Updates to rdma_disconnect, rdma_join_multicast, and return codes.

2007-11-29  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: document retry count limits
	Clarify retry count and RNR retry count limits.  These are 3-bit values.

	Response based on customer feedback.

2007-11-06  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: fix-up man pages
	Fix a couple of errors in the man page documentation and add
	infiniband specific text about QP configuration settings.  This
	is in response to user questions about various settings based
	on feedback from Or.

2007-10-18  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: release 1.0.4

	librdmacm/cma: provide wrapper functions to extract src/dst addresses
	Provide wrapper functions to retrieve the source and destination
	addresses.  This is based on feedback from Doug Ledford.

2007-10-16  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/cma: provide sanity checks for max outstanding rdma ops
	Ensure that the responder_resources and initiator_depth values
	provided by the user are supported by the local hardware.  This
	traps errors sooner during connection establishment (when calling
	rdma_connect), rather than waiting until the modify QP fails
	(after calling rdma_accept).

	librdmacm/man: update man pages to clarify connection request params
	Document connection requests parameters in rdma_connect(),
	rdma_accept(), and rdma_get_cm_event(), specifically regarding
	initiator_depth and responder_resources.

2007-10-01  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: prepare new release

2007-09-24  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/man: update documentation regarding private_data_len
	The private_data_len on the receive side is the size of the data
	buffer, and not the size of the private data sent by the remote side.
	For IB, the size of the sent data is not known, so private_data_len
	is the size of the private data field carried in the CM message.

2007-09-10  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/examples: Add option to set TOS.

2007-09-07  Sean Hefty  <sean.hefty@intel.com>

	rdma/cm: Add ability to specify the type of service
	Add rdma_set_option, similar to setsockopt, to allow the user to
	specify the quality of service provided by a connection.

2007-09-07  Steve Wise  <swise@opengridcomputing.com>

	librdmacm/rping: Persistent rping server.
	Support a rping server mode where the server handles many incoming
	connections by creating threads to process each new rping session.

2007-09-06  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: add valgrind support.

2007-09-05  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/build: minor tweak to build settings for consistency

2007-08-14  Dotan Barak  <dotanb@dev.mellanox.co.il>

	librdmacm: Fix memory leak reported by valgrind
	Fix memory leak reported by valgrind:
	==6239== 16 bytes in 1 blocks are definitely lost in loss record 2 of 10
	==6239==    at 0x4A04CBF: calloc (vg_replace_malloc.c:279)
	==6239==    by 0x4E386C4: ibv_get_device_list@@IBVERBS_1.1 (device.c:65)
	==6239==    by 0x4C2D868: ucma_init (cma.c:221)
	==6239==    by 0x4C2F831: rdma_create_event_channel (cma.c:299)
	==6239==    by 0x4018A5: main (cmatose.c:650)

2007-07-09  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: update for release 1.0.2.

	librdmacm/man: install rdma_event_str man page

2007-07-06  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/README: Update readme file to describe two interface problem

2007-06-28  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/rping: return failure if address/route resolution fails
	If address or route resolution fails, we still return success (0)
	from rping_bind_client.  Return a failure instead.

2007-06-22  Sean Hefty  <sean.hefty@intel.com>

	librdmacm/examples: have sample code display event string
	Have sample code display event strings when an error occurs, rather
	than only displaying the event code.

	librdmacm/cma: add routine to convert rdma cm events to strings

2007-06-21  Or Gerlitz  <ogerlitz@voltaire.com>

	librdmacm/udaddy: remove confusing code from udaddy
	as the man page of rdma_connect, the qp_num and retry_count
	params are relevant only to RDMA_PS_TCP call.

2007-05-23  Sean Hefty  <sean.hefty@intel.com>

	rping: remove need for server to bind to a specific address
	Allow server to listen across all RDMA devices.  Remove -a option
	requirement.

	rping: set default for -p options
	If a port number is not given, use a default, rather than silently failing.

2007-05-17  Sean Hefty  <sean.hefty@intel.com>

	Release 1.0.1

	man: tweak man page format to match other OFED code

2007-05-10  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: fixup .map file
	A couple of APIs listed in the include file are not exposed in the
	.map file, and a couple of older APIs that no longer exist are.  Fix this.

2007-05-09  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: Update build to 1.0.2-rc.

	librdmacm: add man pages

2007-05-04  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: fix typo in comment

2007-05-03  on behalf of Steve Wise  <general-bounces@lists.openfabrics.org>

	rping: Transfer rkey/addr/leninformation in network byte order.
	Fix regression introduced by 88fc0cb21698dfb5d7660eecf7dddd0531fc8021.

	- swizzle memory info when sending it to peer.
	- fixed printf format

2007-04-27  Sean Hefty  <sean.hefty@intel.com>

	Release 1.0

2007-04-26  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: set source port after calling rdma_bind_addr.
	If a user specified port 0 when binding, the actual port number
	assigned was being overwritten before returning from rdma_bind_addr.

2007-04-25  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: update mailing list info

2007-04-25  Steve Wise  <swise@opengridcomputing.com>

	rping: Transfer rkey/addr/len information in network byte order.
	Transfer rkey/addr/len information in network byte order.

2007-04-21  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: update datagram tests to abort if msg size > MTU
	If the user specifies a message size for mckey or udaddy tests that's
	larger than the active MTU of the bound port, the sends will fail.
	Detect this condition and abort the test if the message size is >
	the active MTU of the port.

2007-04-06  Sean Hefty  <sean.hefty@intel.com>

	RDMA/librdmacm: update to rc2

	RDMA/cma: fix 32-bit user / 64-bit kernel mismatch issue
	A 64-bit kernel will pad the size of the event structure to a
	multiple of 8-bytes.  When using a 32-bit kernel, the structure is left
	aligned to a 4-byte boundary.  This results in the userspace event structure
	being too small because of the padding.  Fix this by increasing the padding
	at the end of the userspace event structure.

2007-03-05  Sean Hefty  <sean.hefty@intel.com>

	Fix build when version script is supported.

2007-02-28  Sean Hefty  <sean.hefty@intel.com>

	Add some documentation references to libibverbs calls.

	Enhance API documentation.

2007-02-23  Sean Hefty  <sean.hefty@intel.com>

	Fix RPM release name.

	Update autotools info for 1.0-rc1 release.

2007-01-31  Sean Hefty  <sean.hefty@intel.com>

	Workaround kernel crash when running with ABI 3.
	There's a bug in kernel version ABI 3 that leads to a system crash.
	Avoid calling rdma_init_qp_attr which causes this crash when working with
	ABI 3.

	Update mckey test program to join specific multicast groups.
	Add options to mckey to allow a user to specify a given multicast group.
	This allows the user to join a group using IP address 0, get back the
	actual group MGID that was created for the user, and join that group from
	a separate copy of mckey.

2007-01-26  Sean Hefty  <sean.hefty@intel.com>

	Allow unicast traffic over IPOIB port space.
	Adjust the RMDA_PS_IPOIB to allow unicast traffic.  This requires
	changing how QPs are initialized in order to get the correct qkey
	to use.  We need to call into the kernel to get the initial QP
	attributes.

	Update the udaddy unicast test program to test this capability.

2007-01-24  Sean Hefty  <sean.hefty@intel.com>

	Add support to join IPOIB multicast groups
	Add to the librdmacm an IPOIB port space that allows interoperability with
	IPOIB multicast traffic.  Use of the RDMA_PS_IPOIB is limited to multicast
	join/leave.

	Rename the RDMA_UD_QKEY to RDMA_UDP_QKEY to signify that the qkey is only
	used with the RDMA_PS_IPOIB port space.  Update mckey test program based on
	patch from Or Gerlitz.

2007-01-08  Sean Hefty  <sean.hefty@intel.com>

	librdmacm: return 0 if rdma_leave_multicast is successful.
	Return 0 if leave multicast succeeds, rather than the size of the write.
	Update mckey test program to call rdma_leave_multicast to verify its
	operation.

	Merge branch 'master' into multicast

2006-12-29  Steve Wise  <swise@opengridcomputing.com>

	librdmacm Pass back the status or errno in RDMA CM events.
	The librdmacm code isn't passing back the errno in all events.

	For example, if a connection request times out the kernel CMA will pass
	up event RDMA_CM_EVENT_UNREACHABLE with the status set to -ETIMEDOUT.
	This errno isn't currently passed back to the librdmacm user in the event.

2006-12-15  Sean Hefty  <sean.hefty@intel.com>

	Merge branch 'rdma_ucm-abi3'

	Auto-create config directory during build.

2006-12-12  Or Gerlitz  <ogerlitz@voltaire.com>

	rdma_leave_multicast does not return zero on success
	rdma_leave_multicast does not return zero on success, but rather the size
	of the leave multicast message.

2006-12-05  Sean Hefty  <sean.hefty@intel.com>

	Add multicast support to librdmacm
	Add support for multicast QPs through the rdma cm library.

	Update librdmacm to support ABI version 3
	rdma_ucm ABI 3 adds support for reporting connection information with
	connect events and UDP port space support.  Update the librdmacm to
	take advantage of these features, and update test programs as well.

2006-10-19  Dotan Barak  <dotanb@mellanox.co.il>

	r9898: Compilation warning on 64 bit machines was fixed.

2006-10-03  Steve Wise  <swise@opengidcomputing.com>

	r9696: Add support for iwarp to librdmacm.

2006-09-11  Sean Hefty  <sean.hefty@intel.com>

	r9420: Identify declarations as C, to support C++.

2006-09-05  Sean Hefty  <sean.hefty@intel.com>

	r9272: Update documentation for rdma_destroy_id to call out that it will cancel any oustanding asynchronous operation on the id.  Suggested clarification by Or.

2006-08-31  Sean Hefty  <sean.hefty@intel.com>

	r9208: Remove inlining of sends and set send size = message_size, rather than subtracting the length of the GRH.

2006-08-30  Sean Hefty  <sean.hefty@intel.com>

	r9183: Need to poll both send and receive completions from the CQ on the passive side of the connection.  Without polling the sends first, we exit before receiving all replies.

2006-08-24  Sean Hefty  <sean.hefty@intel.com>

	r9105: Older versions of Linux do not have the misc device.  Backport patches create the RDMA CM abi_version file under /sys/class/infiniband_ucma, so we'll auto- matically look there for the abi_version if we don't find it in its normal location.

	r9104: Command structure is already initialized to zeroes.  No need to set private_data_len field to 0 separately.

2006-08-18  Steve Wise  <swise@opengridcomputer.com>

	r9008: Set the QKEY to a common global value for all UD QPs and multicast groups created by the RDMA CM.  This lets a UD QP be used for datagram communication, as well as multicast group communication over any number of groups.

2006-08-08  Sean Hefty  <sean.hefty@intel.com>

	r8851: Set dst->private_data_len to 0, not src.

2006-08-04  Pete Wyckoff  <pw@osc.edu>

	r8825: Do not type the characters "dst_ip=" explicitly, just the IP address, in both the trunk and iwarp branch.

	r8824: Fix RDMA CM event declaration.

2006-07-24  Sean Hefty  <sean.hefty@intel.com>

	r8647: Re-instate fix from svn 7079, which was removed in svn 7497.
	On kernels 2.6.9 an older, class/misc does not exist.  To support kernel
	backports, if we are unable to locate rdma_cm/abi_version, assume the
	latest version.  If we're wrong, we'll fail later in the code calling into
	the ABI.

2006-07-17  Michael S. Tsirkin  <mst@mellanox.co.il>

	r8548: Offsetof macro needs stddef.h to pull in the definition.

2006-07-14  Sean Hefty  <sean.hefty@intel.com>

	r8523: Add calls to the RDMA CM to return its list of RDMA devices.  The calls are similar to ibv_get_device_list() / ibv_free_device_list().
	Without this patch, RDMA device contexts are handed to the user only after
	they create an rdma_cm_id and bind it to a local device.  By exposing the
	device list to the user, it makes it easier for the user to allocate device
	specific resources (such as PDs, CQs, etc.) that are shared among multiple
	rdma_cm_id's.

	r8517: Remove dependency on libsysfs.

2006-06-26  Sean Hefty  <sean.hefty@intel.com>

	r8215: Enable svn ID on files.

2006-06-23  Hal Rosenstock  <halr@voltaire.com>

	r8187: librdmacm/examples/udaddy.c: Fix example name in messages

2006-06-22  Hal Rosenstock  <halr@voltaire.com>

	r8170: librdmacm/examples/mckey.c: Fix example name in messages

2006-06-21  Dotan Barak  <dotanb@mellanox.co.il>

	r8159: Added checks to the return values of all of the functions that may fail (in order to add this test to the regression system).

2006-06-16  Sean Hefty  <sean.hefty@intel.com>

	r8084: Add support to the userspace RDMA CM library for joining and leaving multicast groups based on IP addressing.  Add a goofy test program for this.

	r8077: Add support for UD QPs to the RDMA CM library, along with a goofy test program

2006-06-13  Pradipta Kumar Banerjee  <bpradipt@in.ibm.com>

	r7968: Ensure that the rping buffer is big enough for the minimum string transmitted.

2006-06-13  Boyd Faulkner  <faulkner@opengridcomputing.com>

	r7966: This patch resolves a race condition between the receipt of a connection established event and a receive completion from the client.  The server no longer goes to connected state but merely waits for the READ_ADV state to begin its looping.  This keeps the server from going back to CONNECTED from the later states if the connection established event comes in after the receive completion (i.e. the loop starts).

2006-06-06  Sean Hefty  <sean.hefty@intel.com>

	r7743: Userspace support to allow overriding the default timeout and retry used by the RDMA CM when connecting over Infiniband.

2006-06-01  Sean Hefty  <sean.hefty@intel.com>

	r7620: We need to ack discarded events to avoid hanging during destruction.

2006-05-25  Roland Dreier  <rolandd@cisco.com>

	r7497: Development versions of libibverbs don't include libsysfs headers or link with libsysfs implicitly any more.  So libibcm and librdmacm need to explicitly include and link libsysfs.

2006-05-18  Steve Wise  <swise@opengricomputing.com>

	r7341: Fixed usage statement.

2006-05-17  Dotan Barak  <dotanb@mellanox.co.il>

	r7298: Added checks to the return values of all functions that may fail - in order to add the test to the regression system.

2006-05-10  Jack Morgenstein  <jackm@mellanox.co.il>

	r7079: On kernels 2.6.9 and back, we didn't find a way to add sysfs attributes to misc devices. If the abi version file does not exist, assume latest ABI to make it possible to use librdmacm on such systems.

2006-05-10  Sean Hefty  <sean.hefty@intel.com>

	r7077: rdma_destroy_event_channel() should be a void, not return an int.

2006-05-09  Sean Hefty  <sean.hefty@intel.com>

	r7021: Replace deprecated function and structure names with updated names.

	r7019: Introduce event channels.  Event channels allow the application to direct how RDMA CM events are reported, by directing events on different rdma_cm_id's to different event channels.  These are similar to completion channels used in verbs.

2006-05-05  Sean Hefty  <sean.hefty@intel.com>

	r6950: Add routines to the user RDMA CM library to get/set transport specific options.
	Add an option to retrieve possible path records for a connection, and set
	which path a connection will be established on.

2006-04-30  Steve Wise  <swise@opengridcomputing.com>

	r6787: Move the state to ERROR if rdma_resolve_route() fails.

2006-04-27  Sean Hefty  <sean.hefty@intel.com>

	r6674: If librdmacm has an error processing an ESTABLISHED event, it will report a CONNECT_ERROR event to the user.  However, the kernel will list the cm_id state as connected.  This can result in librdmacm receiving a DISCONNECTED event on a cm_id that it failed to fully connect from userspace.  This event should not be reported to the user.
	This condition can occur when the remote side establishes a connection, then
	disconnects before the local side processes the ESTABLISHED event.  When the
	ESTABLISHED event is finally processed, the local side will get a failure
	transitioning the QP, causing librdmacm to report CONNECT_ERROR.  Since the
	connection was never reported as ESTABLISHED to the user, do not report the
	DISCONNECTED event.

	A similar situation exists between the CONNECT_RESPONSE and REJECTED events.

2006-04-25  Sean Hefty  <sean.hefty@intel.com>

	r6628: Update to support calling listen without having previously bound.  After listen is called, we need to query for the route information to know which port has been assigned.

2006-04-24  Steve Wise  <swise@opengridcomputing.com>

	r6593: Fixed X86_64 compile warnings.

2006-04-18  Sean Hefty  <sean.hefty@intel.com>

	r6503: Remove ABI check surrounding call to query route information after performing a local bind.  We always need to get the route information in order to bind to the local device, or DAPL breaks.

2006-04-08  Sean Hefty  <sean.hefty@intel.com>

	r6335: Reset ABI version to 1.  We will not maintain backwards compatibility until the RDMA CM has been merged upstream.  This removes the ABI version hack that tried to handle the case where the ABI version was not exported into userspace.

2006-04-07  Steve Wise  <swise@opengridcomputing.com>

	r6319: Handle the case on the server side where the first RECV completion happens before the connect ESTABLISHED event.

2006-04-06  Bryan O'Sullivan  <bos@pathscale.com>

	r6297: update librdmacm.spec.in from 1.0 branch.

	r6296: update librdmacm.

2006-04-03  Dotan Barak  <dotanb@mellanox.co.il>

	r6183: Added a print if post send fails.

2006-04-03  Tziporet Koren  <tziporet@mellanox.co.il>

	r6171: merge cmatose example from main trunk

2006-03-31  Sean Hefty  <sean.hefty@intel.com>

	r6135: If the user calls rdma_bind_addr(), but specifies either a zero IP address or the loopback address, bind will succeed, but the cm_id will not be attached to an RDMA device.  This will result in a failure if rdma_resolve_addr is called.  Fix rdma_resolve_addr(), so that it handles this condition properly.
	Also fix the userspace interface to allow querying for address information
	after binding to the zero or loopback address.  This breaks the behavior of
	the current ABI, so bump the ABI version, and add the proper support to
	allow checking the kernel ABI version from the userspace library.

2006-03-29  Sean Hefty  <sean.hefty@intel.com>

	r6087: Permit specifying the source address of a connection as a command line parameter.

2006-03-20  Sean Hefty  <sean.hefty@intel.com>

	r5906: Fix dev name.  Should be 'rdma_cm' instead of 'ucma'.

2006-03-07  Bryan O'Sullivan  <bos@pathscale.com>

	r5663: first cut at packaging fun for 1.0-rc1.

2006-02-24  Bryan O'Sullivan  <bos@pathscale.com>

	r5492: Branching for 1.0 release

2006-02-11  Steve Wise  <swise@opengridcomputing.com>

	r5368: Implement an RDMA ping-ping test over an RC connection.  This program uses send, recv, RDMA read, and RDMA write operations, as well as CQ channels to get CQ events.

2005-12-22  Sean Hefty  <sean.hefty@intel.com>

	r4579: Add 'void' to function prototypes to make gcc happy.

2005-12-14  Roland Dreier  <rolandd@cisco.com>

	r4466: Change libibverbs API for listing all known devices from ibv_get_devices() to ibv_get_device_list(), and update all in-tree uses of this API.

2005-11-29  Sean Hefty  <sean.hefty@intel.com>

	r4186: Initialize file operations if user tries to get events or retrieve the fd before creating a cm_id.

2005-11-24  Sean Hefty  <sean.hefty@intel.com>

	r4128: Fix setting source/destination addresses.

2005-11-17  Sean Hefty  <sean.hefty@intel.com>

	r4074: Defer initialization from library load until user allocates their first rdma_cm_id.

2005-11-16  Sean Hefty  <sean.hefty@intel.com>

	r4067: And now export the rdma_get_fd function...

	r4065: Provide call to retrieve fd.

2005-11-15  Sean Hefty  <sean.hefty@intel.com>

	r4050: Fix wildcard bind_addr call.  No device is associated with a wildcard bind, so no route is available.

2005-11-11  Sean Hefty  <sean.hefty@intel.com>

	r4022: Add port_num field to struct rdma_cm_id to make it easier for clients to identify which port they are currently using.
	Remove a few unneeded variable initializations, and fix a bug where the
	IB address was not set after calling rdma_bind_addr() in userspace.

	r4019: Add support for userspace RDMA connection management abstraction (CMA).
	- Adds common user/kernel data structures and copy routines in ib_uverbs and
	  libibverbs.
	- Update ib_ucm and libibcm to use common structures and copy routines.
	- Add new kernel module, rdma_ucm, to support userspace CMA.
	- Add userspace CMA library, librdmacm.
	- Add userspace CMA test program.
	- Update kernel CMA to support userspace CMA library.
		- Add support for users to transition QP states.
		- Add backlog parameter to rdma_listen API.
