previous         next         contents

Appendix C: Solutions for frequent problems

Problems:

• Skip text until keyword/symbol
• Avoid heading spaces, comments,... in SOURCETEXT
• Two-pass translation
• Individual test of productions in complex environments
• Tuning branch selection
• Make linefeed ended comments
• Copy comments from source to target
• Copy and process comments
• Reentrance and Multithreading

---

Skip text until keyword/symbol

Sometimes one has to process/translate only parts of the source, which start with special symbols, keywords, or something else.

ExampleSkipExcl =
 { ('%%'  | any); !c#1; }
-> SOURCETEXT.

ExampleSkipIncl =
 c:= 2;
 {!c#1; ('%%'  | any) }
-> SOURCETEXT.

ExampleSkipExcl stops in front of the keyword, ExampleSkipIncl skips the keyword, too.
If there is a keyword to be found, one should add a branch:

ExampleSkipExcl =
 { ('REPEAT' $ | ident | any); !c#1; }
-> SOURCETEXT.

Otherwards it would be possible that the parser stops already on something like NOREPEAT.

A call to Skip in front of large alternative may also speed up the parser if it is likely to have to skip many characters at this point.

back to list

Avoid heading spaces, comments,... in SOURCETEXT

As SOURCETEXT means an exact copy of the appropriate source text stretch there is no way to exclude comments, spaces, newlines etc. within a construct from being copied. However, you can avoid such insignificant characters heading the construct. This is done by invoking the intrinsic procedure Skip() before entering the respective rule.

idcp = ident -> SOURCETEXT.
withhd = idcp -> idcp_.
withouthd = Skip(); idcp -> idcp_.

Then withhd will translate "    alfa" into "    alfa", while withouthd produces just "alfa".

back to list

Two-pass translation

This can be achieved easily by writing the target out on a file and including this file afterwards. If this is to slow, one can use internal buffering by replacing Dp4Streams with Dp4StrBuf.
There is no need to limit this to two passes.

  IMPORTS Dp4Streams, Dp4Chars, Dp4DocFiles;
ExampleTwoPasses =
  ExampleFirstPass;
(* write 1st target of ExampleFirstPass on file: *)
  Dp4Streams.Tar2Strm(ExampleFirstPass_, Dp4DocFiles.New("Temp1File"));
(* make this file the source for further processing: *)
  From(Dp4DocFiles.IFile("Temp1File"));
  ExampleSecndPass;
-> ExampleSecndPass_.

ExampleFirstPass = ident -> ident_ '1st'.
ExampleSecndPass = ident -> '2nd Pass:' ident_.

back to list

Individual test of productions in complex environments

Selecting a certain nonterminal as root will no longer work if global declarations are involved. (very likely in any real application)
Here is a proposal to overcome this problem:
Create a special rule that establishes the needed environment and then calls ident4root. Test your productions by putting their name in front of the source.

Example

TestRoot=
  GLOBVAR DCL globvar1, globvar2,....;  (* declarations *)
  globvar1:= ...                        (* initializations - if needed *)
  ident4root                            (* calls the wanted production *)
-> ident4root_ .                        (* gives the result *)

Lets say, you want to test your rule MyExpr, then select TestRoot as root and write as source

MyExpr a+(B*Pi)/128

and the result will be quite similar to selecting MyExpr as root in a simple grammar without global variables.

back to list

Tuning branch selection

If branch selection is determined by heading keywords it can be speeded up by applying the following translation principle:

Instead of

 ('keyword1' $ ... |'keyword2' $ ... |'keyword3' $... )

one should better use

 ident
  (!ident_0='keyword1'...|!ident_0='keyword2'...|!ident_0='keyword3'...)

This avoids touching the source repeatedly.
(Do not expect too much. More often than not this tuning will not result in sensible speed-ups. If there is realy a big number of keywords, i.e. >20...40, and the selection is touched really frequently, one should think about using an importet hash table. Thus, replace the second line by (/My.hash(Ident_0)/ ...|...|...) )

One can achieve a nominal speed-up by placing a Skip() in front of alternatives. But be careful, this may interfer with suppression of skipping (<...> constructs).

back to list

Make linefeed ended comments

There are programming languages where a line end also closes comments (e.g. Ada, C++). One can produce such comments simply with help of the line terminal.

makeComment = { line[i] }
-> {'// ' line_[i] }.

This rule describes a translation which converts any text in a C++ comment format.

back to list

Reentrance and Multithreading

Generally, Depot4's translators are reentrant and, as long as no imported or external components are involved, thread safe.

Reentrance

One can perform a nested translation by use of the From/Back mechanism (see example).
In principle, the instantiation of a separate parser by use of Dp4Tools.translate is possible too.

Multithreading

Starting from version 1.9, multithreading is possible. The data structures for the management of global data and parsing objects (cp. substitution of implementations) are stored in a Translator object now. To avoid changes to existing translators the previous interface is maintained by help of a (widely hidden) default such object. To exploit the enhancement one has to explictely use a dedicated translator object for each thread.
However, the runtime does not handle any locking/sysnchronization issues when translators from different threads want to access the same object.

back to list

    previous         next         contents