Why To Source Code

Why compile to Scheme source code?

• one can see what is going to be executed,
• it is possible to rewrite parts of the source code to make some sorts of optimization,
• in some cases it is possible to translate Scheme source code into the other programming languages,
• smart Scheme compiler can optimize code while compiling it,
• it allows a sort of AOP.

Is Scheme Faster than C?

Article by Jonathan Sobel. Author wrote a program in Scheme, then automatically transformed it into another form, and the result outperformed a lot of C/Assember competitors at least twice.

On note from Kirill Lisovsky

(his letter, my answer)

If we parse XPath to something intended for rewriting/optimization (or even transformation to another language, in your case), then this intermediate representation has no need to be an evaluatable Scheme code...

I agree with it.

OTH, some Scheme code may be used for AST representation (SXPath uses S-expressions). Even more: it's possible to design such an AST in a fashion which will allow it's direct evaluation - it may generate the query-processing code, for example.

It is very near to what I have done. My S-expressions are Scheme programs on the one side, and AST-like expressions on the others side.

Notes from Oleg Kiselyov

(his letter)

Isn't it like the specialization of the general sxpath function of the type

sxpath:: SXPath -> SXML -> Nodelist

to the particular value of SXPath? Then we are talking about Partial Evaluation (PE for short) -- which is a vast and developed area. I wonder why Michael Sperber hasn"t said anything: he wrote several well-cited papers on this topic (as well as a partial evaluator for the full Scheme, right Mike?). You can see the extensive bibliography at http://library.readscheme.org/page10.html

...

So, xpath->scheme can generate open code (which has free variables), and sxlet will generate code that captures those variables with bindings. Hmm, sounds a bit risky. Dealing with open code is quite error-prone and difficult. Perhaps we should consider the type of sxpath to be

sxpath:: SXPath -> Root -> xOtherVar -> SXML -> Nodelist

or

sxpath:: SXPath -> xOtherVarDict -> SXML -> Nodelist

and specialize with respect to SXPath and xOtherVarDict. ...

Notes from Dmitry Lizorkin

(my letter, his letter)

There is an advantage of Scheme source code over AST. AST is fixed and so it can't provide additional functionality. Scheme source code is flexible, so we can do tricks.

For example, consider XPath "elem{position()<3]".

...

In case of the Scheme code it's possible to change "scxpath-filter" to some "take-until". In case of the AST we can't perform such change without extending AST.

The same optimization can be implemented above AST as well. Moreover, this optimization is already implemented. Please take a look at the function

ddo:check-special-predicate

in "ddo-txpath.scm"

Also note that AST is SXML, so it can be processed by conventional SXML tools.

Another issue is documentation. Scheme code is documented (R5RS) much better than AST. (Is AST documented at all?)

The grammar for AST is as follows:

... skipped ...

The mapping from XPath grammar to AST isn't documented yet. On the other hand, I believe that the code in "xpath-ast.scm" that performs the mapping considered is self-describing enough.

A sort of AOP

(my letter)

Use of "sxlet" allows to make local mofidications to XPath execution.

Current SCXPath is used as follows:

(define tree "(*top* (AAA (BBB) (CCC) (BBB) (BBB) (DDD (BBB)) (CCC))))
(define xpath-str "/AAA/CCC")

(define sc1 (xpath->scheme xpath-str))
(define sc2 (sxlet ((*root* tree)) sc1))
(define x (eval sc2 (interaction-environment)))
(pp (x tree))

First, I create a Scheme code from an XPath string, then I bind variables using the "sxlet" macro. Having done that, I use "eval" to make a lambda and finally I apply the lambda to the tree. Result is as expected:

((CCC) (CCC))

But macro "sxlet" not only defines variables, it also can redefine procedures. Example:

(define sc2-b (sxlet (
    (*root* tree)
    (sxml:child
      (lambda (test-pred?)
        (lambda (nodeset)
          ((node-trace "child axis (out)")
            ((sxml:child test-pred?)
              ((node-trace "child axis (in)") nodeset)))))))
  sc1))
(define x-b (eval sc2-b (interaction-environment)))
(pp (x-b tree))

Now the transcript looks so:

-->child axis (in) :(*top* (AAA (BBB) (CCC) (BBB) (BBB) (DDD (BBB)) (CCC)))
-->child axis (out) :((AAA (BBB) (CCC) (BBB) (BBB) (DDD (BBB)) (CCC)))
-->child axis (in) :(AAA (BBB) (CCC) (BBB) (BBB) (DDD (BBB)) (CCC))
-->child axis (out) :((CCC) (CCC)) ((CCC) (CCC))

So I've affected XPath evaluation without changing Scheme code. It is not a superior archievement, but it may be quite useful.