Implementing Rogue's Module/Namespace System
I decided to add "modules" (AKA namespaces) to Rogue to be able to start making some auxiliary libraries without worrying about name collisions with end-developer code.
Here's the syntax - pretty straightforward and C++-esque:
# AlphaBeta.rogue $module Alpha class Value ... endClass class XYZ ... endClass $module Beta class Value ... endClass
# Test.rogue $include "AlphaBeta.rogue" println Value() # doesn't work println Alpha::Value() # creates an Alpha::Value $using Alpha println Value() # creates an Alpha::Value $using Beta println Value() # creates a Beta::Value; later 'using' commands can # shadow the namespace of earlier commands println XYZ() # creates an Alpha::XYZ because there is no Beta::XYZ
I first tried implementing this module system at the Parser level with limited success. There was always some edge case that complicated things or that I had missed.
I redid the system at the Tokenizer level and it proved to be a simple, elegant solution. Here's an overview of the steps I took:
Before implementing modules my files were compiled in the following order: tokenize a file, preprocess the file tokens, parse the file tokens, repeat for the next file.
I reordered my monolithic compile to be a slightly different order: tokenize every file, then preprocess every file's tokens, and finally parse every file's tokens.
::being a unique symbol reserved for being a module name separator, I modified the Tokenizer to accept
::as part of an identifier, so writing
Alpha::Valueis turned into a single identifier token.
During token preprocessing of a particular file I took note of every
$module ModuleNamedirective and used
ModuleName::as the module context for any class definitions that followed.
When I encountered any
class Xdefinition during preprocessing (all Rogue type definitions begin with
class), I collected the class name "X" into a central lookup table as an identifier declared under the
ModuleNamenamespace. After I was done I might have collected the following in essence:
I then added a new step. In-between preprocessing every file's tokens and then parsing every file's tokens I added a call to
insert_module_prefixes(). This just runs through every file's tokens one more time and performs the following steps:
If there is a
$module Xor a
$using Xthen I add all of module X's declared identifiers into a current
module_id_maptable. The example table in Step 5 above would produce the following module id map when added:
Whenever I see an identifier token, I look up its name in the
module_id_mapand substitute any mapped ID I find there. Like C++ I accept a leading
::on an identifier as a way to escape out of module namespace and back into default namespace.
println Beta::Valueby the time those tokens hit the parser. If an identifier is already prefixed with
Alpha::then it doesn't occur in the
module_id_mapand so it's fine.
One implication of this technique is that you could sprinkle
::in your identifiers quite arbitrarily and independently of the module namespace system. I don't see that as a bad thing; just interesting.