WGML control word separator implementation problems

[Jiri Malak]

Hi,

I got into problem with control word separator processing, it looks like
inplementation is strange on multiple processing level.
below is example which demonstrate a problem

.sr prf$='test'
.cw $
.sr cnt_pfx='&prf$.'
.cw ;

line
.sr cnt_pfx='&prf$.'

is broken into two lines
.sr cnt_pfx='&prf
.'

it is wrong, prf$ is macro and is used for text expansion that it should
not be broken by control word separator
in this case line should not be broken to two lines.

it looks like that this line division is done by calling
process_line(); // substitute variables + functions

but should be done by calling scan_scr(), where it should be context
sensitive

Jiri

[Paul S Person]

I agree that it is a theoretical problem, but does it affect the docs?
If not, can it be fixed simply and easily, without affecting anything
else? If it requires extensive work, it might be better to wait.

But there is hope: while wgml 4.0 accepts your example and puts "test"
in both symbols (.ty is your friend!), if you don't change the control
word separator but simply use it you get an error:

Error! 70: Expecting an equal sign
Line 76 of file '..\docs\plain.gml'

.sr prf
*
(our version, but wgml 4.0's is similar), which means that, in some
sense, wgml 4.0 regards control word separator ";" as /different from/
"$"-when-used-as-a-control-word-separator in this context, whatever it
may regard "this context" as being.

This suggests a simple test
(CW_sep_char == CW_SEP_CHAR_DEFAULT)
at the appropriate point so that the character is only treated this
way when it is, in fact, ";". And /that/ might be a simple fix that
doesn't affect anything else, but how can you tell if you don't check
for new diffs resulting from the change?

Post by Jiri Malak
Hi,
I got into problem with control word separator processing, it looks like
inplementation is strange on multiple processing level.
below is example which demonstrate a problem
.sr prf$='test'
.cw $
.sr cnt_pfx='&prf$.'
.cw ;
line
.sr cnt_pfx='&prf$.'
is broken into two lines
.sr cnt_pfx='&prf
.'
it is wrong, prf$ is macro and is used for text expansion that it should
not be broken by control word separator
in this case line should not be broken to two lines.
it looks like that this line division is done by calling
process_line(); // substitute variables + functions
but should be done by calling scan_scr(), where it should be context
sensitive
Jiri

[Paul S Person]

On Thu, 06 Jun 2019 17:30:23 -0700, Paul S Person

Post by Paul S Person
But there is hope: while wgml 4.0 accepts your example and puts "test"
in both symbols (.ty is your friend!), if you don't change the control

should have been

[Jiri Malak]

I don't think the simple solution.
Because line is divided before it is processed by scan_scr().
It will require much deeper change and appropriate code is shared by
multiple logical part as GML tag processing etc.

Anyway it is used in all Error Description documents as wccerrs.gml,
wpperrs.gml etc.

Post by Paul S Person
On Thu, 06 Jun 2019 17:30:23 -0700, Paul S Person

Post by Paul S Person
But there is hope: while wgml 4.0 accepts your example and puts "test"
in both symbols (.ty is your friend!), if you don't change the control

should have been

[Paul S Person]

Post by Jiri Malak
I don't think the simple solution.
Because line is divided before it is processed by scan_scr().
It will require much deeper change and appropriate code is shared by
multiple logical part as GML tag processing etc.

It is my understanding that Frank Beythien followed the documented
procedure in the Reference Manual 8.1 Processing Rules, as modified by
actual experience, and the documentation of control word .cw in the
script-tso, in relevant part:

"If substitution (.SU) of Set Symbols is ON and the input line is a
control-word line, the substitution scan will terminate at the
first control word separator. The remainder of the line will be
substituted when it is processed later as an input line."

This pretty much means that the behavior you observed is "correct"
insofar as the documentation goes. What you have discovered is a
deviation from the documented behavior on the part of wgml 4.0.

Diving briefly into this code, which I have made a few modifications
to, but not very many, I noticed a "ProcFlags.CW_sep_ignore" which, if
it were set when a symbol evaluation has been identified early enough
in the process, might solve the problem. Or not, as the case may be.

But, no, like most of these problems, the solution is /not/ to tear
out large chunks of code and start over. The solution is to find
/where the problem is/ and /fix it/. And the fix itself is usually
fairly simple; the hard part is figuring how to do it.

I have no doubt that, once wgml is stable (does all the docs correctly
and works properly within the build system), some, many, or all of the
various subsystems could do with being torn out and rewritten.
Preferably after thinking about it a bit. I have several such possible
projects in mind myself, for some of my early decisions have produced
some incredibly obtuse code. But not unti wgml is stable, so that
regressions can be caught easily and fixed.

Post by Jiri Malak
Anyway it is used in all Error Description documents as wccerrs.gml,
wpperrs.gml etc.

Then it will have to be dealt with when I get to those documents, if
you don't want to do it now. I can't blame you for not wanting to;
figuring out what to change and where to change is likely to be rather
harder than other problems I have encountered.

Post by Jiri Malak

Post by Paul S Person
On Thu, 06 Jun 2019 17:30:23 -0700, Paul S Person

Post by Paul S Person
But there is hope: while wgml 4.0 accepts your example and puts "test"
in both symbols (.ty is your friend!), if you don't change the control

should have been

[Paul S Person]

On Fri, 07 Jun 2019 10:38:36 -0700, Paul S Person
<cut to chase>

Post by Paul S Person

Post by Jiri Malak
Anyway it is used in all Error Description documents as wccerrs.gml,
wpperrs.gml etc.

Then it will have to be dealt with when I get to those documents, if
you don't want to do it now. I can't blame you for not wanting to;
figuring out what to change and where to change is likely to be rather
harder than other problems I have encountered.

Sooner than that, quite likely: bld\cc\gml\msgmacs.gml both uses ".cw
$" and symbol names with "$", so I may need to fix this for cguide
with PS!