Board index FlightGear Development Nasal

String manipulation and file I/O (pm2thread)

Nasal is the scripting language of FlightGear.

String manipulation and file I/O (pm2thread)

Postby Hooray » Sat Sep 12, 2015 9:39 am

Hooray wrote:
Thorsten wrote:
  • what do we have in terms of string manipulation, i.e. cut out a part of a string, cast it into an integer, or such things? I remember we could test whether a string contains a phrase, but somehow that doc also got moved.
  • do we have anything on file access, i.e. could we read a data file into a vector somehow?


just briefly for now (may respond in more detail later on):

the general API reference (for the main library itself) is this: http://plausible.org/nasal/lib.html
(this is also located somewhere in $FG_ROOT/Docs IIRC)

That should get you going with basic string manipulation (str, int, substr etc)
For more sophisticated stuff, refer to $FG_ROOT/Nasal/canvas/std/string.nas (IIRC, look for a file named string, which is loaded into the std namespace)

There is a http.nas module doing fancy templating stuff using string manipulation.

Equally, for file access, there is the "io" namespace/module - but you can directly use the FG wrappers in $FG_ROOT/Nasal/io.nas which are pretty high-level.

io.nas also contains helpers for XML handling, so your file could even be arbitrary XML or property list xml markup, which supports vectors etc trivially.

For flat data files (CSV), io.readfile() is simple enough to use

I guess you could repost the question and get a more detailed response in the Nasal forum, including code snippets/example, i.e. when you tell us what you want to do.

Once we enough material, we could turn the corresponding thread into dedicated wiki articles (haven't checked if there is anything yet or not)

2-3 concrete questions and use-cases should be enough to provide code snippets that we can turn into examples for the wiki
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: String manipulation and file I/O (pm2thread)

Postby Hooray » Sat Sep 12, 2015 9:43 am

Another possibility would be using a subset of JSON, i.e. to serialize/unserialize Nasal data structures as scalars, vectors and hashes - which should be pretty straightforward. The format would then be plain text, and it would just be evaluated via Nasal's compile()/call() APIs to turn data structures back into Nasal types.

This would have the added advantage that the serialization format could even support expressions in the form of:

Code: Select all
[ 1, 2, 4 +math.pi, sqrt(1024) ]


Obviously, you could then also directly use io.load_nasal() to load a Nasal file from disk (think mydata.nas) and for serialization purposes, write to the same file - just make sure that it is valid Nasal
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: String manipulation and file I/O (pm2thread)

Postby Hooray » Sat Sep 12, 2015 4:40 pm

I just noticed that the original Nasal example, already contains a helper function to serialize a Nasal type:

http://plausible.org/nasal/sample.nas
Code: Select all
##
## A rockin' metaprogramming hack.  Generates and returns a deep copy
## of the object in valid Nasal syntax.  A warning to those who might
## want to use this: it ignores function objects (which cannot be
## inspected from Nasal) and replaces them with nil references.  It
## also makes no attempt to escape special characters in strings, which
## can break re-parsing in strange (and possibly insecure!) ways.
##
dump = func(o) {
    result = "";
    if(typeof(o) == "scalar") {
        n = num(o);
        if(n == nil) { result = result ~ '"' ~ o ~ '"'; }
        else { result = result ~ o; }
    } elsif(typeof(o) == "vector") {
        result = result ~ "[ ";
        if(size(o) > 0) { result = result ~ dump(o[0]); }
        for(i=1; i<size(o); i=i+1) {
            result = result ~ ", " ~ dump(o[i]);
        }
        result = result ~ " ]";
    } elsif(typeof(o) == "hash") {
        ks = keys(o);
        result = result ~ "{ ";
        if(size(o) > 0) {
            k = ks[0];
            result = result ~ k ~ ":" ~ dump(o[k]);
        }
        for(i=1; i<size(o); i=i+1) {
            k = ks[i];
            result = result ~ ", " ~ k ~ " : " ~ dump(o[k]);
        }
        result = result ~ " }";
    } else {
        result = result ~ "nil";
    }
    return result;
}
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: String manipulation and file I/O (pm2thread)

Postby Thorsten » Sat Sep 12, 2015 5:19 pm

Thanks.

Let me perhaps give the context (one pic for two points...):

Image

Some of the displays of the Shuttle contain ideal trajectories. These would be mission-specific though, so it'd be ultimately cool to just swap data files to get the displays adjusted. Right now the trajectory data is hard-coded into the Nasal controller for the display, but I'd rather be more elegant.

The second point is seen below - above the menu structure in fact. The menu structure is something belonging to the glass cockpit era of the Shuttle fleet. But above the line is the 'old' way of interacting with the computer, and that still is the majority of displays.

Here, commands are entered using a keypad, for instance a particular ops mode, say 201 is called by keying in a sequence OPS 201 PRO. That'd be easy enough to do - but also parameters can be set by sequences like ITEM 4 + 20.1 +70.0 - 30.5 EXEC which uses + and - as delimiters as well as signs and is shorthand for 'set item 4 to 20.1, item 5 to 70.0 and item 6 to -30.5'.

So I'm guessing I need a parser for this kind of sequence, for which I need the ability to slice the command string into its logical parts and convert them to numbers.

Right now I'm just looking at what is available so that I can formuate a plan of attack for the thing.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: String manipulation and file I/O (pm2thread)

Postby Hooray » Sat Sep 12, 2015 6:05 pm

For the command interface, you would probably want to use a stack (the data structure) and implement a parser using a so called "state machine", which can be as simple as a switch/case statement (in C/C++), or simply a hash with command tokens as lookup keys to register callbacks for each command:

Code: Select all
var CommandMgr = {

'EXEC': func(state, operands) {
},

'OPS': func(state, operands) {
},


'+': func(state, operands) {
},

'-': func(state, operands) {
},

};



A simple parser could then look for each key in the CommandMgr hash (which is just a container for your callbacks, i.e. not using any OOP as is) - and if it finds a registered key, it would call the callback, passing in the current state of the state machine, including any operands the command may have (e.g. a vector of operands).

Obviously, this could also use classes/objects for better encapsulation - or do without any hashes at all, i.e. using just explicit if/else statements if you prefer a more verbose version.

For parsing purposes, you would basically tokenize the string (e.g. looking for valid tokens/separators) and then use a simple parser to call the callback for each token (possibly recursively, and possibly taking operator precedence into account)

by sequences like ITEM 4 + 20.1 +70.0 - 30.5 EXEC which uses + and - as delimiters as well as signs and is shorthand for 'set item 4 to 20.1, item 5 to 70.0 and item 6 to -30.5'.

in that case, you would need a simple stack, so that you know that subsequent + instructions also increment the item index

This kind of thing can be trivially implemented in Nasal, the main keywords are "parsing", "tokenize" and "state machine". However, there already is code in SimGear (C++) that implements an abstract state machine, so another option would be exposing/using that code - then again, performance is likely to be negligible for something like that.

For examples on writing parsers in Nasal you could refer to io.nas (xml parsing) or svg.nas (parsing svg/xml using Nasal).

I would need to look at the trajectory data to see what kind of serialization scheme would work best, but I guess I would suggest to go with conventional PropertyList XML files, because they are easy to process (create, write and read) without having to bother with complex logic like representing vectors or trees, because those are naturally mapped to XML space using the PropertyList subset.

Equally, Nasal hashes can be trivially mapped to PropertyList space and vice versa using the props.nas module, e.g. see the Canvas "menubar" snippet on the wiki, which demonstrates how to load an existing property list XML file into Nasal space and process it without having to do any explicit parsing, simply by mapping XML keys to keys in a Nasal hash:

http://wiki.flightgear.org/Howto:Making ... nubar#Code
Code: Select all
load: func() {
  var fgroot_path = getprop("/sim/fg-root");
  var path = fgroot_path ~ '/' ~ me._filename;
 
  # next, load menubar.xml using read_properties() (see io.nas) :
  var menubar = io.read_properties( path );
 
  # now, turn the whole thing into a Nasal hash for dead-simple processing
  me.menubar = menubar.getValues();
 },


In this case it is using OOP (me keyword), but what it will do is pretty straightforward:
  • assemble a path relative to fgroot
  • append the file name using a member field
  • call io.read_properties() to load the file and save it in a property object
  • turn the tree into a Nasal hash that can be directly processed using a foreach loop

As can be seen in the build() method, there is no explicit parsing involved here, because each XML tag will become Nasal key, with sub-tags becoming child nodes in the hash. Identical nodes (same level) will be turned into vector elements, i.e. can be accessed using foo[0]..foo[x]

BTW: Could you post links to the corresponding Nasal code so that we can take a look ?
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: String manipulation and file I/O (pm2thread)

Postby Hooray » Sat Sep 12, 2015 8:23 pm

Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU

Re: String manipulation and file I/O (pm2thread)

Postby Thorsten » Wed Sep 16, 2015 5:38 pm

Thanks for the pointer with the state machine - that turns out to be much easier to do than a full parser on the final string. I don't think a simple lookup hash ever does the trick, because there's cross dependence of what things mean based on what OPS mode is on and such things, so the conditionals get a bit awkward, but the general principle applies very efficiently.
Thorsten
 
Posts: 12490
Joined: Mon Nov 02, 2009 9:33 am

Re: String manipulation and file I/O (pm2thread)

Postby Hooray » Wed Sep 16, 2015 6:52 pm

a hash with lookup keys/callbacks should work pretty well as long as you pass a "state" variable, i.e. a vector serving as a stack with operands, which is basically how the XML/SVG parsers are implemented.
Please don't send support requests by PM, instead post your questions on the forum so that all users can contribute and benefit
Thanks & all the best,
Hooray
Help write next month's newsletter !
pui2canvas | MapStructure | Canvas Development | Programming resources
Hooray
 
Posts: 12707
Joined: Tue Mar 25, 2008 9:40 am
Pronouns: THOU


Return to Nasal

Who is online

Users browsing this forum: No registered users and 2 guests