ircd: Various comments added/modified.

README.md

@@ -8,10 +8,11 @@ made its way to William Pitcock et al, whom after 2005 developed the project und
 *Charybdis*.
 
 In 2014 a protocol was proposed to reinvigorate real-time communication in lieu of growing
-commercial competition and a lack of innovation from open source alternatives to
+proprietary competition and a lack of innovation from open source alternatives to
 compete. This protocol is known as the **Matrix protocol**.
 
-**IRCd now implements the Matrix protocol.**
+**IRCd now implements the Matrix protocol** using some of the latest techniques available
+for modern C++ free software.
 
 
 # Charybdis/5

include/ircd/README.md

@@ -32,14 +32,11 @@ design decision for making IRCd easier to understand for contributors.
 
 The library is based around the `boost::asio::io_service` event loop. It is still
 an asynchronous event-based system. We process one event at a time; developers must
-not block execution. Events are never processed concurrently on different threads✝.
+not block execution. Events are never processed concurrently on different threads.
 
-However, there are some ✝'s here which must be addressed. We have introduced
-additional standard threads to libircd with the purpose of "offloading" operations
-from some library dependencies that don't cooperate asynchronously. This ensures the
-"main thread" running the actual event loop is never blocked in any case. Furthermore,
-some 3rd party dependencies like RocksDB (and boost::asio's DNS resolver) may
-introduce threads into the address space which they handle privately.
+✝ If there is ever a truly long-running computation or a call to a 3rd party
+library which will do IO and block the event loop, we may use an additional
+`std::thread` to "offload" this operation.
 
 ##### libircd introduces userspace threading
 

include/ircd/buffer.h

@@ -324,7 +324,8 @@ struct ircd::buffer::const_raw_buffer
 };
 
 /// fixed_buffer wraps an std::array with construction and conversions apropos
-/// the ircd::buffer suite
+/// the ircd::buffer suite. fixed_buffer should be punnable. Its only memory
+/// footprint is the array itself and
 ///
 template<class buffer,
          size_t SIZE>

include/ircd/ctx/context.h

@@ -51,7 +51,6 @@ namespace ircd::ctx
 /// complete. If the child context does not cooperate the destructor will hang.
 /// To prevent this behavior `detach()` the ctx from this handler object; the
 /// detached context will free its own resources when finished executing.
-/// This is bad practice except in certain cases.
 ///
 /// To wait for the child context to finish use `join()`. Calling `interrupt()`
 /// will cause an `interrupted` exception to be thrown down the child's stack

include/ircd/fs.h

@@ -37,6 +37,12 @@
  */
 
 /// Tools for working with the local filesystem.
+///
+/// IRCd has wrapped operations for the local filesystem to maintain a
+/// cross-platform implementation of asynchronous file IO in conjunction with
+/// the ircd::ctx userspace context system. These operations will yield your
+/// ircd::ctx when necessary to not block the event loop on the main thread
+/// during IOs.
 namespace ircd::fs
 {
 	IRCD_EXCEPTION(ircd::error, error)
@@ -94,6 +100,7 @@ namespace ircd::fs
 	void chdir(const std::string &path);
 	bool mkdir(const std::string &path);
 
+	// This suite of IO functions may yield your context.
 	std::string read(const std::string &name);
 	string_view read(const std::string &name, const mutable_raw_buffer &buf);
 	bool write(const std::string &name, const const_raw_buffer &buf);

include/ircd/json.h

@@ -24,39 +24,6 @@
 
 /// JavaScript Object Notation: formal grammars & tools
 ///
-/// The IRCd JSON subsystem is meant to be a fast, safe, and extremely
-/// lightweight interface. We have taken a somewhat non-traditional approach
-/// and it's important for the developer to understand a few things.
-///
-/// Most JSON interfaces are functions to convert some JSON input to and from
-/// text into native-machine state like JSON.parse() for JS, boost::ptree, etc.
-/// For a parsing operation, they make a pass recursing over the entire text,
-/// allocating native structures, copying data into them, indexing their keys,
-/// and perhaps performing native-type conversions and checks to present the
-/// user with a final tree of machine-state usable in their language. The
-/// original input is then discarded.
-///
-/// Instead, we are interested in having the ability to *compute directly over
-/// JSON text* itself, and perform the allocating, indexing, copying and
-/// converting entirely at the time and place of our discretion -- if ever.
-///
-/// The core of this system is a robust and efficient abstract formal grammar
-/// built with boost::spirit. The formal grammar provides a *proof of robust-
-/// ness*: security vulnerabilities are more easily spotted by vetting this
-/// grammar rather than laboriously tracing the program flow of an informal
-/// handwritten parser.
-///
-/// Next we have taught boost::spirit how to parse into std::string_view rather
-/// than std::string. Parsing is now a composition of pointers into the original
-/// string of JSON. No dynamic allocation ever takes place. No copying of data
-/// ever takes place. IRCd can service an entire request from the original
-/// network input with absolutely minimal requisite cost.
-///
-/// The output side is also ambitious but probably a little more friendly to
-/// the developer. We leverage boost::spirit here also providing *formally
-/// proven* output safety. In other words, the grammar prevents exploits like
-/// injecting and terminating JSON as it composes the output.
-///
 namespace ircd::json
 {
 	IRCD_EXCEPTION(ircd::error, error);

include/ircd/json/README.md

@@ -0,0 +1,36 @@
+## JavaScript Object Notation
+
+##### formal grammars & tools
+
+The IRCd JSON subsystem is meant to be a fast, safe, and extremely
+lightweight interface. We have taken a somewhat non-traditional approach
+and it's important for the developer to understand a few things.
+
+Most JSON interfaces are functions to convert some JSON input to and from
+text into native-machine state like JSON.parse() for JS, boost::ptree, etc.
+For a parsing operation, they make a pass recursing over the entire text,
+allocating native structures, copying data into them, indexing their keys,
+and perhaps performing native-type conversions and checks to present the
+user with a final tree of machine-state usable in their language. The
+original input is then discarded.
+
+Instead, we are interested in having the ability to *compute directly over
+JSON text* itself, and perform the allocating, indexing, copying and
+converting entirely at the time and place of our discretion -- if ever.
+
+The core of this system is a robust and efficient abstract formal grammar
+built with boost::spirit. The formal grammar provides a *proof of robust-
+ness*: security vulnerabilities are more easily spotted by vetting this
+grammar rather than laboriously tracing the program flow of an informal
+handwritten parser.
+
+Next we have taught boost::spirit how to parse into std::string_view rather
+than std::string. Parsing is now a composition of pointers into the original
+string of JSON. No dynamic allocation ever takes place. No copying of data
+ever takes place. IRCd can service an entire request from the original
+network input with absolutely minimal requisite cost.
+
+The output side is also ambitious but probably a little more friendly to
+the developer. We leverage boost::spirit here also providing *formally
+proven* output safety. In other words, the grammar prevents exploits like
+injecting and terminating JSON as it composes the output.

include/ircd/json/array.h

@@ -40,7 +40,7 @@ namespace ircd::json
 /// iterating this array by incrementing your own numerical index and making
 /// calls into this object is NOT efficient. Simply put, do not do something
 /// like `for(int x=0; x<array.count(); x++) array.at(x)` as that will parse
-/// the array from the beginning on every single iteration. Instead, use the
+/// the array from the beginning on every single increment. Instead, use the
 /// provided iterator object.
 ///
 struct ircd::json::array

ircd/ctx.cc

@@ -90,6 +90,10 @@ ircd::ctx::ctx::ctx(const char *const &name,
 {
 }
 
+/// Base frame for a context.
+///
+/// This function is the first thing executed on the new context's stack
+/// and calls the user's function.
 void
 ircd::ctx::ctx::operator()(boost::asio::yield_context yc,
                            const std::function<void ()> func)
@@ -123,6 +127,10 @@ noexcept
 		func();
 }
 
+/// Direct context switch to this context.
+///
+/// This currently doesn't work yet because the suspension state of this
+/// context has to be ready to be jumped to and that isn't implemented yet.
 void
 ircd::ctx::ctx::jump()
 {
@@ -146,6 +154,13 @@ ircd::ctx::ctx::jump()
 	interruption_point();
 }
 
+/// Yield (suspend) this context until notified.
+///
+/// This context must be currently running otherwise bad things. Returns false
+/// if the context was notified before actually suspending; the note is then
+/// considered handled an another attempt to `wait()` can be made. Returns true
+/// if the context suspended and was notified. When a context wakes up the
+/// note counter is reset.
 bool
 ircd::ctx::ctx::wait()
 {
@@ -168,6 +183,10 @@ ircd::ctx::ctx::wait()
 	return true;
 }
 
+/// Notifies this context to resume (wake up from waiting).
+///
+/// Returns true if this note was the first note received by this context
+/// while it's been suspended or false if it's already been notified.
 bool
 ircd::ctx::ctx::note()
 {
@@ -178,6 +197,7 @@ ircd::ctx::ctx::note()
 	return true;
 }
 
+/// Wakes a context without a note (internal)
 void
 ircd::ctx::ctx::wake()
 try
@@ -189,6 +209,8 @@ catch(const boost::system::system_error &e)
 	ircd::log::error("ctx::wake(%p): %s", this, e.what());
 }
 
+/// Throws if this context has been flagged for interruption and clears
+/// the flag.
 void
 ircd::ctx::ctx::interruption_point()
 {
@@ -196,6 +218,8 @@ ircd::ctx::ctx::interruption_point()
 		throw interrupted("ctx(%p) '%s'", (const void *)this, name);
 }
 
+/// Returns true if this context has been flagged for interruption and
+/// clears the flag.
 bool
 ircd::ctx::ctx::interruption_point(std::nothrow_t)
 {
@@ -216,12 +240,17 @@ ircd::ctx::ctx::interruption_point(std::nothrow_t)
 //
 __thread ircd::ctx::ctx *ircd::ctx::current;
 
+/// Yield the currently running context until `time_point` ignoring notes
 void
 ircd::ctx::this_ctx::sleep_until(const std::chrono::steady_clock::time_point &tp)
 {
 	while(!wait_until(tp, std::nothrow));
 }
 
+/// Yield the currently running context until notified or `time_point`.
+///
+/// Returns true if this function returned because `time_point` was hit or
+/// false because this context was notified.
 bool
 ircd::ctx::this_ctx::wait_until(const std::chrono::steady_clock::time_point &tp,
                                 const std::nothrow_t &)
@@ -233,6 +262,10 @@ ircd::ctx::this_ctx::wait_until(const std::chrono::steady_clock::time_point &tp,
 	return std::chrono::steady_clock::now() >= tp;
 }
 
+/// Yield the currently running context for `duration` or until notified.
+///
+/// Returns the duration remaining if notified, or <= 0 if suspended for
+/// the full duration, or unchanged if no suspend ever took place.
 std::chrono::microseconds
 ircd::ctx::this_ctx::wait(const std::chrono::microseconds &duration,
                           const std::nothrow_t &)
@@ -243,11 +276,12 @@ ircd::ctx::this_ctx::wait(const std::chrono::microseconds &duration,
 	const auto ret(c.alarm.expires_from_now());
 
 	// return remaining duration.
-	// this is > 0 if notified or interrupted
+	// this is > 0 if notified
 	// this is unchanged if a note prevented any wait at all
 	return std::chrono::duration_cast<std::chrono::microseconds>(ret);
 }
 
+/// Yield the currently running context until notified.
 void
 ircd::ctx::this_ctx::wait()
 {
@@ -256,6 +290,9 @@ ircd::ctx::this_ctx::wait()
 	c.wait(); // now you're yielding with portals
 }
 
+/// Post the currently running context to the event queue and then suspend to
+/// allow other contexts in the queue to run. Until we have our own queue the
+/// ios queue makes no guarantees if the queue is FIFO or LIFO etc :-/
 void
 ircd::ctx::this_ctx::yield()
 {
@@ -274,18 +311,25 @@ ircd::ctx::this_ctx::yield()
 	while(!done);
 }
 
+/// Throws interrupted if the currently running context was interrupted
+/// and clears the interrupt flag.
 void
 ircd::ctx::this_ctx::interruption_point()
 {
 	return cur().interruption_point();
 }
 
+/// Returns true if the currently running context was interrupted and clears
+/// the interrupt flag.
 bool
 ircd::ctx::this_ctx::interruption_requested()
 {
 	return interruption(cur());
 }
 
+/// Yield to context `ctx`.
+///
+///
 void
 ircd::ctx::yield(ctx &ctx)
 {
@@ -305,6 +349,7 @@ ircd::ctx::yield(ctx &ctx)
 	notify(ctx);
 }
 
+/// Notifies `ctx` to wake up from another std::thread
 void
 ircd::ctx::notify(ctx &ctx,
                   threadsafe_t)
@@ -315,12 +360,17 @@ ircd::ctx::notify(ctx &ctx,
 	});
 }
 
+/// Notifies `ctx` to wake up. This will enqueue the resumption, not jump
+/// directly to `ctx`.
 bool
 ircd::ctx::notify(ctx &ctx)
 {
 	return ctx.note();
 }
 
+/// Executes `func` sometime between executions of `ctx` with thread-safety
+/// so `func` and `ctx` are never executed concurrently no matter how many
+/// threads the io_service has available to execute events on.
 void
 ircd::ctx::signal(ctx &ctx,
                   std::function<void ()> func)
@@ -328,6 +378,9 @@ ircd::ctx::signal(ctx &ctx,
 	ctx.strand.post(std::move(func));
 }
 
+/// Marks `ctx` for interruption and enqueues it for resumption to receive the
+/// interrupt which will be an exception coming out of the point where the
+/// `ctx` was yielding.
 void
 ircd::ctx::interrupt(ctx &ctx)
 {
@@ -335,36 +388,42 @@ ircd::ctx::interrupt(ctx &ctx)
 	ctx.wake();
 }
 
+/// Indicates if `ctx` was ever jumped to
 bool
 ircd::ctx::started(const ctx &ctx)
 {
 	return ctx.started();
 }
 
+/// Indicates if the base frame for `ctx` returned
 bool
 ircd::ctx::finished(const ctx &ctx)
 {
 	return ctx.finished();
 }
 
+/// Indicates if `ctx` was interrupted; does not clear the flag
 bool
 ircd::ctx::interruption(const ctx &c)
 {
 	return c.flags & context::INTERRUPTED;
 }
 
+/// Returns the notification count for `ctx
 const int64_t &
 ircd::ctx::notes(const ctx &ctx)
 {
 	return ctx.notes;
 }
 
+/// Returns the developer's optional name literal for `ctx`
 ircd::string_view
 ircd::ctx::name(const ctx &ctx)
 {
 	return ctx.name;
 }
 
+/// Returns a reference to unique ID for `ctx` (which will go away with `ctx`)
 const uint64_t &
 ircd::ctx::id(const ctx &ctx)
 {