Towards an infrastructure for examples

Request For Comments
#1

Lack of examples is perceived as biggest problem with the SWI-Prolog documentation. Writing examples is a lot of work and the same example is often relevant to multiple predicates. I’ve added a feature to the website to manage examples outside the manual and have them automatically appearing with all the predicates they use and refer to.

See e.g. append/2 or assert/1.

The examples themselves are for now in a git repository, where the README roughly explains how it works.

As is, changes to the repository reach the backends within 5 minutes and the CDN in one hour. Note that you can simply add and edit files on GitHub and thus we can give people the authority to add examples directly.

Please share your opinion. Would this work? What can be improved? What procedure do we need to ensure good examples? Can we write a guide as to what a good example is?

Please do not start making zillions of PRs before this infrastructure is stable. Notably using a Discourse wiki category might be an option to simply discussion around examples.

7 Likes
About the Example - Discussion category
About the Example category
#2

FWIW, when I worked on IBM Prolog, we wrote two manuals: reference manual and user’s guide. Both of them had code snippets; we had a workflow that stripped the code snippets from the documentation, ran them, and verified that the results were as expected (this also gave us a nice set of test cases).
[Sadly, I didn’t keep copies of the manuals, and I can’t find them online.]

2 Likes
#3

This technique which has been coined ‘doctests’ nowadays sees wide usage in a number of other programming languages (c.f. Elixir, Rust and Python).
From first-hand experience with being tasked to write maintainable code in each of these languages, I would like to give this technique my glowing recommendation.

EDIT: it seems like there was already some prior discussion on this forum about this some 1.5 years back: Does PLDoc have a way to incorporate unit tests akin to Python's doctest? although I do not know in which way (if at all) PLdoc and SWI-Prolog’s documentation capabilities are related.

3 Likes
#4

That is a bit of a disjoint topic. It touches ones an important related topic though: should we tag the code sections with enough information to make this possible and -related- to allow for an embedded SWISH instance?

To be able to run an example you need to make sure you have all relevant program text, some of which you may not (by default) show in the example. You may also need context such as present files, environment variables, access to (web-)services, etc. For SWISH you may not be able to run the example due to sandbox restrictions.

We can deal with most of that using the ```tag syntax, e,g. ```swish

#5

If the documentation depends on a SWISH instance being available, I don’t think that’s a good thing; the documentation ought to be available offline. If the SWISH instance allows for playing around with the examples without having to cut&paste, that’s fine of course.

I would presume that any “unit tests” that are shown as part of the documentation could have an invisible part (setup) and visible part (example predicate definitions, queries, and results). For SWISH example, I presume that both the setup and visible part from the documentation would be available for modification.

#6

Surely, having a SWISH instance around will just be an add-on for the website for examples where that makes sense. How is open to debate.

As is, examples handled this way are only available online, i.e… on https://www.swi-prolog.org. At some point some of this (notably the primary example file) could be included into the offline docs. First of all I want to get to a workflow that get us high quality examples in machine readable form. What exactly we do with them besides showing them in the online help is secondary.

edit An alternative way to make examples available might be SWISH. We do not have effective protection, discussion and quality support for content on SWISH though.

#7

One major question I always have when writing examples is should the example be simple enough for someone in their first Prolog class or should it be presented as part of a real world example. For examples of use by a student in their first Prolog class, the JavaDoc examples are what I am thinking. For real world examples I am thinking along the lines of a programming language cookbook.

The guide address things like, should examples be pure. IMHO they should not as most beginners would just be confused by it. If however there are cookbook topics that clearly identify they are for a pure solution, then I am fine with that. I am not against pure solutions as I do use them from time to time.

When I learned to program for my degree, the data center library had all of the needed IBM manuals and they were one of the best hidden gem’s for learning. (Way before the Internet existed as it does now.) I firmly agree that there should be separate reference manuals (SWI-Prolog reference manual) and user manuals, which I see as something different from a cookbook recipe. The phrase that helped me most to understand what a user manual was when someone called them by the name abusers manual. Not that I like that name, but it does get the point across that the manual gives you enough info to even learn how to abuse a method/predicate, etc.

Another point that has not been addressed is that there should be template to follow for each type of item in a document.

#8

Hello Jan, this is a really good start IMHO. I like how the examples are both prominent but don’t get in the way until you click on them.

As a first step, I will share a couple of existing examples from the docs that I consider good examples of how to write such examples (hmm :wink: ).

There is the split_string/4 examples: https://www.swi-prolog.org/pldoc/doc_for?object=split_string/4

There is the small example about “Checks for a character to be in a set” at the bottom here: https://www.swi-prolog.org/pldoc/man?section=ext-dquotes-port. This one is interesting because it touches upon a lot of features of Prolog/SWI-Prolog.

There is the example from read_pending_codes/3. It also touches many interesting topics (repeat/0, fill_buffer/1, the use of double negation).

One nice to have example would be why, when, and how to use when/2 with ?=/2. Throughout the years I can recall using when/2 twice, and both times I thought I am hacking and not developing software. I should maybe try to dig those out and first share them here on Discourse to discuss them before adding examples to the git repository?

#9

That is always hard to say in general. Every user has different tastes and needs. The good thing is that by separating examples from the reference manual text is that we can analyze them, associate them dynamically with reference manual sections, order them and add icons to indicate what they are. It is probably quite easy to tell real world examples apart from “just this predicate” examples as they involve more code and more predicates. It is also fairly easy to figure out code is complete or not by cross-referencing it and find undefined predicates and syntax errors. Possibly we needs tags. I think we can get a long way without them though.

It is a delicate subject :frowning:

Please do. I don’t want huge numbers of examples before the details are settled to avoid a lot of work moving the entered examples to the final format and storage location. So, please concentrate on diversity to get an idea about what we want.

#10

I actually did share the examples. I meant existing examples that you have already written in the docs (this is where the links are pointing to).

So, to rephrase: “Are the examples behind those links good examples of writing code examples?”

#11

I think they are. As I wrote most of them this isn’t very objective :slight_smile: I wonder whether we should remove them from the manual and stick them into the example infrastructure? That is a bit too early, but would that be a good step for the future?

#12

I created a new category for just the examples, and a separate subcategory for the related discussions.

While the discussions will work as expected where users can post new topics, add replies, etc. for the examples the thought is that each topic in the example should be a single post that can be moved into the PR with no to little editing. With the Wiki category topics they are all Discourse Wiki post meaning anyone with a high enough trust level can edit the post. Currently as the options are set, only the creator of the post and admins will be able to edit it.

As this could lead to a land grab should all of the topics in the category example also be a Discourse Wiki so that those with high enough trust level can edit them?

Originally when I created some Wiki post, I was the owner of the topic. Latter I changed all of them to the user Wiki so that other users would not think they were imposing on a user by editing such a topic. Should the post in the category Example also have those changed to a non-user owner?

#13

Hmmm. Isn’t it a little too early to call between using github and Discourse? We surely do not want both. Each of them does have advantages though. Github has good versioning and ownership tracking and reasonable reviewing. Also getting files in and out of github is easier. Discourse has better discussion and likes that we could use to rank examples. I don’t know which would encourage more contributions. Opinions?

#14

I think we are thinking different things here.

After reading your first post I thought that you wanted to get the official documentation to have more examples and that those examples would eventually be added to the official documentation via the PR process. But to avoid the zillions of PRs, the example can start off here with Discourse and be discussed. Along with the discussion the example would be created and modified in an ongoing process. Once it seemed reasonable a user could then create the PR so the example would become part of the official documentation. Apparently I am misreading the intent.

So tossing out that idea, then it seems that if somehow we create an example in Discourse it gains a link to the documentation as demonstrated with assert/1. But I don’t see how what we create here gets referenced in the documentation. So now I am confused.

#15

I guess the trick is to create a good workflow. So yes, there are good examples resulting from discussions here. How to go from there?

  1. Create the example using a wiki-like setup on Discourse and you’ve created.
    a. Leave them there and use the Discourse API to get them into the manual
    b. Create a GitHub PR from it.
  2. Create a GitHub PR directly and use GitHub code reviewing and PR handling to fix stuff.

It seems you have 1+b in mind. I was thinking about either 1+a or 2. Technically they can all work. The choice is mainly on what will work better.

1 Like
#16

Should examples show the modes for each predicate?

I know that understanding the mode and what they mean should cut down on lots of questions, but if a user has trouble understanding a simple predicate, then trying to understand modes for a predicate might just make a bad situation worse.

Perhaps with simple and common predicates like append/3, the most common use should be demonstrated then in a latter section more complex examples demonstrated. I know this use append/3 what not something I would thought of at first.

Never underestimate append/3:

subst(This, That, MyStr, Result) :-
    append(This, After, Rest),
    append(Before, Rest, MyStr),
    !,
    subst(This, That, After, AfterResult),
    append([Before,That,AfterResult], Result).
subst(_, _, S, S).

To test:

103 ?- portray_text(true).
true.

104 ?- subst(`this`,`that`,`athishellothis`,R).
R = `athathellothat`.
#17

This is rather complicated. I’d vote for keeping it simple for append/3 and have an example on list substitution. That will get linked automatically.

I’ve illustrated this. See https://eu.swi-prolog.org/pldoc/doc_for?object=append/3

Note that the examples are in the wrong order. I’ll fix that.

#18

I not trying to throw wrenches into your plan for examples, which I really do like, I am just trying to address big issues, (test cases, comments in code, complex examples, etc.), now rather than latter. By setting the tone now it should be easier along the way. :slightly_smiling_face:

1 Like
#19

Dear Jan,

My long held opinion is that SWI is let down by documentation issues.
There are great features in the system that are not used as much as they deserve to.

These issues are on both the small and large scale. Large, as in large-picture; a road map of
interesting features. Examples come under the small-scale category. I guess it will
be good to have a separate infrastructure, but my opinion is the manual should
be strengthened not weakened, before that happens.

I rarely go to the on-line manual because I run my local doc server which
serves all my local packs in addition to the system and packages predicates.
(Script spudlike.pl in pack(spuds).)

The one simple thing i long felt was missing is a single example for each
predicate in the docs. When we write code, it is obvious what we mean,
until we have to use the predicate a few months later.
This is important especially as there is a fragmented nomenclature in the
community and the manual.

Regards,

Nicos Angelopoulos

1 Like
#20

@nicos I certainly share your sentiment. One work-around I have used is to look at uses of predicates within the library source code. I even had an idea of trying to somehow annotate those but never got far due to having things to do :frowning: This approach is of course not always an option.