How is goal expansion for closures defined?

SWI-Prolog web site and services
#1

I am trying to find a documentation of the following behaviour. If I enter this code:

goal_expansion(println(X), (write(X), nl)).
test :- call(println, 'Hello World!').

Listing shows me this result:

test :-
    call('__aux_wrapper_8a89205eca9a6ffb31dd01cc968a2aa022fa1f49', 'Hello World!').

'__aux_wrapper_8a89205eca9a6ffb31dd01cc968a2aa022fa1f49'(A) :-
    write(A),
    nl.

Is this always the case that closures are first uncurried, then expanded as normal goals, and then curried again via an auxiliary predicate?

#2

Goal expansion on a meta argument marked 1…9 adds 1…9 variables to the goal (creating println(X)) and then expands this. If the output of the expansion is a simple goal with the same variables at the end we simply delete these. Else we create a wrapper predicate. Cute, no?

#4

That is more an artifact of the toplevel which tries to resolve predicates first (and proposes to fix typos, lacking module qualifiers, etc). In any case, I think goal expansion has been a mistake. In due time this will be replaced by providing inline predicates. That guarantees consistent behavior in the various calling modes, makes code analysis a lot easier, doesn’t suffer from dubious interaction with the module system and is easier for the user.

1 Like
#6

(>>)/2 is a predicate.

#8

I have a side question about goal_expansion for the expanding query, which in fact I remember I asked a couple of times. That is, goal_expansion for the top lever query does not work on MacOS. As far as I remember an answer was a limitation of the implementation of goal_expansion at that time. As a long time have passed, I ran the following query expecting the problem has been fixed silently. But still it seems not work as I expected.
IMHO, it seems very confusing that goal_expansion does not work on expanding the query.

% swipl
Welcome to SWI-Prolog (threaded, 64 bits, version 8.3.15-9-gb74ddfb9c-DIRTY)
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
Please run ?- license. for legal details.

For online help and background, visit https://www.swi-prolog.org
For built-in help, use ?- help(Topic). or ?- apropos(Word).

?- [user].
|: goal_expansion(f(X), writeln(X)).
|: ^D% user://1 compiled 0.01 sec, 1 clauses
true.

?- f(hello).
ERROR: Unknown procedure: f/1 (DWIM could not correct goal)
?-
#9

This is the same problem as raised by @j4n_bur53: the toplevel first tries to relate all subgoals to actual predicates and then applies goal expansion. This is done to perform the correct (module sensitive) goal expansion.

As said, goal expansion is IMO, in retrospect, a bad idea. There is nothing/very little you cannot achieve by writing or generating normal predicates. When performance is the reason for goal expansion, partial evaluation/inlining is the clean solution. That route avoids all these problems and makes the language simpler.

#10

Thank you for reply.

I don’t see exactly why goal_expansion is bad idea. But as a user, I have to trust in the developer’s opinion. Anyway what I would like to have is simple, which is something “query_expansion” to expand only top-level query. Of course the user should be responsible for the expanded goal including module resolution. I think it is natural for the user to use his private macros in the top level query. In fact, I have large complex private macros (on closure, regex, conditiona equations, etc). As the goal_expansion does not work to expand my macros in the query, I had to use the expand_query/4 to expand them by way of my term_expansions. As I did not think I asked too much as for
the goal_expansion in my sense, I was wondering what deep things are behind this seemingly simple request.

#11

Hi Jan, could you point to a resource to read up on this or explain very briefly what you mean? Would that be solved in the implementation or is it a change to how the code would look like? Is goal expansion in this sense different from term expansion?

1 Like
#12

I tried to explain that. Goal expansion is poorly related to modules and it is generally extremely hard to apply it in all places where you would expect it. Consider e.g.,

?- maplist(prove, [Goal1, Goal2, ...]).

One solution to all this is to perform the goal expansion as a very last step in dynamic calling. SICStus does that, but it seriously slows down dynamic calling.

For the toplevel is interferes with DWIM (Do What I Mean) in the sense that we must correct goals and fix modules before doing goal expansion … unless the users wants to expand a goal not related to a predicate first. But, nothing can tell these two cases reliably apart.

Just defining predicates do not have these problems. What makes it impossible to use predicates?

Quite easy. If we have @j4n_bur53’s

println(X) :- write(X), nl.

And we somehow think that is too slow, we can simply replace a call to println(X) by write(X), nl instead of calling the predicate. That is not much more than getting the clause and replacing the call with the clause body. In addition we can use possible known arguments to only include matching clauses and we may specialize calls, e.g., a call(X) with a known X simply becomes X. This isn’t completely trivial. We must respect the scope of cuts. We must deal with recursion, for example changing a maplist/N call into a specialized recursive call or a sequence of calls if one of the lists is given.

This way we do not have all the problems of goal expansion. We can do the transformation on user request using e.g. :- inline println/1. by default under certain optimization levels or even lazily based on hotspot observation.

#13

So basically it will “just work” for the client code, and the client code will not need goal expansion at all. There might be either optional annotations (your :- inline directive example) or it can be done by the compiler, and by the virtual machine.

Thank you for the clear explanation.

#14

I am sorry but it is still unclear for me, because I think my question is not related to modules and the DWIM, but I think the “(query) goal_expansion” is just “term_expansion” applied to the query as a prolog term. In fact, I use expand_query/4 to expand my private macros in the query by preparing this lines (following your advice):

user: expand_query(X, Y, Z, Z) :- user:chk_pac_query,
	'$current_typein_module'(C),
	pac:expand_query(C, X, Y).

To ask differently, it would be enough for me if the goal_expansion gives an equivalent support for just this expand_query/4. My long experience says this is convenient for testing queries by typing short, for example.

#15

I’m sorry, but we need to give up DWIM correction for that to work reliably and I think that is the wrong choice, especially as expand_query/4 provides a work around.

1 Like
#17

The source is public :slight_smile:

#19

How would you implement things library(yall) without goal_expansion/2?

I agree that it would be nice to generalize library(apply), but it’s too painful to use term_expansion/2 for expanding individual goals … would you instead suggest a scaffolding for making term_expansion/2 easy to use by handling meta-predicates/

#23

I was thinking about goals within a predicate, not top-level queries.

In older Prologs, there was only term_expansion/2, and no goal_expansion/2. This was fine for things like DCGs, but painful for doing something like inline expansion of goals within a predicate definition because the expansion had to recursively process meta-predicates (and potentially do different things in the context of “and” or “if-then-else”). As an alternative to goal_expansion/2, for transformation that go beyond inlining, some kind of framework for dealing with meta-predicates might be better than goal_expansion/2 (which iterates until it reaches a fix-point).

#26

If I understand you correctly, the problem is that you’d like the top-level query to be processed like

gensym(q, Q), % generate unique predicate name
expand_term(_query_(...) QQ),
assertz((Q:-QQ,write_answer_and prompt(...)), 
call(Q)

but instead there’s a DWIM step instead of expand_term/2.

#30

I don’t really see how this should be different from inline/partial evaluation of the predicates that define library(yall) except for one thing: as we have seen, the semantics of yall wrt shared variables is different between calling yall as a predicate or translating it through goal expansion. I think that is a pretty undesirable feature of this library.

If I recall well, in the goal-expanded version only variables in {X,Y...} are shared and in the called version all variables are shared. It is hard to see another possibility for the normal call. So, I think the correct solution is to stick with the share-all model and be consistent. The only alternative I see is to add some notation that allows for a separate variable scope inside a Prolog term. In fact, SWI-Prolog has that as quasi quotations. We could make this work:

?- maplist({|lambda(X,Y)||Y is X+1|}, [1,2,3], X).

Here, the content of the quasi quotation is parsed independently and the parse can bind variables provided in the part before the ||.

2 Likes
#32

This is pretty much on topic. It explains the problems with goal expansion.

In due time goal_expansion/2 will be replaced by providing inline predicates
How would you implement things library(yall) without goal_expansion/2
#34

In boot/expand.pl. You should be able to figure out how that works with a bit of reading and possibly a bit of experimentation. I’d have to do the same as it is all long ago and grew rather organically. As I think it was a mistake from the start I’m not too keen actually doing so. I’d rather spent my time doing it right.