[Nod]

In an effort to patch up the grey areas of the docs, would it be alright if I "interviewed" you, Peri, on different subjects?

(that is, toss up some batches of questions for you to answer :)

-Nod-

[mpah]

Good idea - but I will be out of circulation for about 10 days from Monday. If you post some questions soon I'll try to produce at least preliminary answers befor then..

Anyway it's good to have you back on the case.
_________________
The Language Machine - a toolkit for language and grammar

[Nod]

Ok then. We'll start with the built-in scripting language. Note that I'm only hammering out questions with little regard to what's already in the docs. Feel free to flame the redundancies :)

*Which primitive data types can be created? Value, reference, map, array? Others? What is the syntax to create each?
*How are variables implemented internally? Is each variable instance its own class instance?
*Can aggregate types be created? Structures/classes? Syntax?
*Which control structures are supported?
*Are named functions supported? Syntax? Anonymous functions?
*Is the language dynamically typed? Can one concatenate a number to a string?
*Which operators are supported? D-style array concat ~ enation? Bitwise?
*Which types of string literals are supported?
*Which escape sequences are allowed?
*Garbage collection or explicit variable destruction?
*What is the syntax to interface with external C/D functions?
*To what extent can one currently interface with C/D? Is it possible to pass any/all types of variables in/out?
*How does the parser distinguish what is rules and what is script?
*How does one perform exception handling? :)

Heh, that's it for now, but I'm sure I'll think of more. And it's good to be back.

-Nod-

[mpah]

== A note about scripting feature in the language machine ==
Conventional scripting features in the language machine are deliberately kept to an absolute minimum. The system is intended as a way of adding grammatical capabilities to existing systems and languages.

On the other hand, you can also say that scripting is everywhere in the language machine - it exists entirely as a system of rules, where the left- and right- sides of rules (after an explicit or implicit initial symbol) are active symbol sequence generators. In effect the left- and right- sides of a rule are scripts or code bodies that produce values for parsing. The left-side produces values that are to be treated as goal symbols, the right-side produces values that are to be treated as input symbols.

The complete source text of the metalanguage frontend is provided as the definitive statement of what should be recognised as vaild - this is the horse's mouth as a recursive experience, and may not be to everyone's taste. To try it, see [http://languagemachine.sourceforge.net/lmn2xfe.html the metalanguage source].

== Which primitive data types can be created? ==
*numbers - underlying type is D language double
*strings and symbols - underlying type is D language string - most strings are unique within a namespace
*lexical classes - a lexical class is a special kind of symbol
*character - underlying type is D language dchar
*sequences - underlying type is D language row of elements
*tables - underlying type is D language associative array

Note that several different namespaces are used:
*system symbols - symbols as used in the grammar
*user symbols
*variable symbols
*system primitives - internal use only

Syntax for variables and values
*variables usually start with upper-case letter - These Are Variables
*an identifier within an expression is treated as a variable
*variables are explicitily declared by the 'var' notation
*the ":" mechanism is used to bind values to variable names during analysis

== How are variables implemented internally?==
*all lmn values are instances of element
*var derives from element and is used for variables
*reference and array reference objects are derived from var and exist during evaluation
*reference objects and array reference objects can be passed to external procedures

== Can aggregate types be created? ==
*the only aggregate type is the table - an array indexed by element
*as in the lua language, 'A.field' is equivalent to 'A["field"]'
*but the ":" notation allows you to construct and communicate compound and structured values

== Which control structures are supported? ==
*conditionals - essentially as javascript/C/D
*loops - "for" and "while" - essentially as javascript
*"do" and "foreach" are not yet implemented

== Are named functions supported? Anonymous functions? ==
*a bracketed element sequence { ... } is like an anonymous procedure
*but these are not functions in any ordinary sense of the word
*function call expressions such as f(a,10) compile as calls on extension procedures

== Is the language dynamically typed? Can one concatenate a number to a string? ==
*the language is dynamically typed without implicit coercions
*a sequence produces a sequence of values - in effect a concatenation
*explicit concatenation and other string operations do not fit easily into grammatical analysis
*the format() extension permits you to produce a string by combining values
*the toChars() extension produces a sequence of character elements

== Which operators are supported? D-style array concat ~ enation? Bitwise? ==
*see http://languagemachine.sourceforge.net/lmn2xfe.html#expressions
*array concatenation is not builtin, but could be added - but see disclaimer at top

== Which types of string literals are supported? ==
*double quoted strings are compiled as matchable symbols
*single-quoted strings are compiled as character element sequences

== Which escape sequences are allowed? ==
*the intention is to allow the same escape sequences as the D language
*there may well still be some omissions in this area

== Garbage collection or explicit variable destruction? ==
*the language machine relies on the D language for garbage collection
*a variable belongs to a context (a left- side nesting in the diagram)
*at present the "all" mechanism relies on a list of all variables
*the "!" operation can be used to truncate the "all" variables list

== What is the syntax to interface with external C/D functions? ==
*just use the familiar notation: f(x, y, z)
*see examples: http://languagemachine.sourceforge.net/extendcalc.html
*all external procedures are effectively variadic
*the actual call sequence depends on how the ruleset is compiled
*see http://languagemachine.sourceforge.net/builtin.html
*the mapped interface passes a row of elements
*the direct interface passes elements as variadic arguments

== Does the C/D interface work? Is it possible to pass any/all types of variables in/out? ==
*the C/D interfaces are fully implemented
*the direct D procedure interface is the easiest and most efficient
*all arguments are passed as elements - you can call their methods
*variable and array reference arguments are not automatically dereferenced
*the toVal function/method is used for dereferencing
*there are methods for assigning to reference elements

== How does the parser distinguish what is rules and what is script? ==
*each side of a rule (after the explicit or implicit initial symbol) is a code body
*a purely syntactic distinction is made between "pattern" and "elements"
*rule left-sides and bracketed sequences are parsed as "elements"
*rule right-sides are analysed as "pattern"
*injected material is analysed as "pattern": ... { a b c <- injected stuff } ...
*expressions and side-effects are permitted only in "elements"
*to use expressions and side-effects in a "pattern", enclose them in braces
*in effect the rules for "elements" recognise statements within element sequence

As an example, the left-side of
alpha beta A = thisismyexternal(A, I++); gamma <- thing;

is compiled as
the initial symbol "alpha"
the symbol "beta"
an assignment: A = thisismyexternal(A, I++);
the symbol "gamma"

It might be clearer to put a semicolon before the assignment, but it is not necessary.

The distinction between "elements" and "pattern" is required because there would otherwise need to be some explicit way of separating the right-side of a rule from the left-side of a following rule.

==How does one perform exception handling? ==
*get back into the D language?
*unlike normal programming, grammatical analysis is about success and failure
*everything to the left of the current position has been successfully matched so far
*if a mismatch occurs and there are rules that may deal with it, the rules are tried in an inner context
*if the mismatch cannot be resolved, the current rule is abandoned and alternative rules are tried
*if there are no alternatives in the present context it failed to resolve the mismatch that triggered it
*so the failure propagates to enclosing contexts until an alternative analysis succeeds

Abandoning the current rule and trying an alternative is in many ways like throwing an exception, except that the system also resets the state of the analysis as far as possible to the state that obtained at the start of the rule - this includes throwing away variables and bindings that have been created since the start of the rule. However the system does not try to track and undo side-effects that change values in enclosing contexts..

There may well be a case for adding exception handling to the language so as to trap errors produced from side-effect code. At present illegal variable references can produce unpretty messages, partly because I was unsure how they ought to be handled.

I hope that helps a bit
Peri
_________________
The Language Machine - a toolkit for language and grammar

[mpah]

You may have noticed that I used mediawiki markup instead of BBCode above - I've also copied the material to a page on the wiki at sourceforge. This makes it easier to add new material to the static pages, which are written using the mediawki conventions.

Of course I can also write a ruleset to turn BBCode into wiki markup or html - in fact I started, but I 'm not likely to get it done immediately.
_________________
The Language Machine - a toolkit for language and grammar