Multi session / multi-tenant web app with SWI-Prolog

SWI-Prolog web site and services
#1

Dear community,
I would to operate ski-prolog to offer a scalable multi-tenant web application.
I know swish (very nice). I know about the web service which can be started from an SWI-prolog instance running on a machine. So, I hope I did not oversee any similar case, my question is:

How can a tomcat-like web application service on the base of prolog be configured and operated?

E.g. I imagine a new user,… logging in, then computing a prolog program, then computing another one, and all in his/her web-session-object. On the server side this session bound environment should be handled (tomcat does it for Java applications).

I am new to swi-prolog and all its power. (I am an old c-prolog fan, which after 30 years came back to origins :wink:

Thank you for any hints / advice on this subject
Kind regards

1 Like
#2

Not really sure what you are up to. Possibly Pengines do what you like. They are the underpinning of SWISH, but without the boilerplating. A Pengine is bound to an HTTP session, provides an isolated computational unit (a thread) and privates storage (a temporary module). The storage can contain both data and code (as this is the same in Prolog :slight_smile: )

The new Web Prolog is more or less the same with a better design but less mature.

1 Like
#3

When you - from a browser - open a web application served by a tomcat, the latter creates an application object in memory (could be an instance of prolog) and sends the session string (sessionid) to the browser. The browser with that sessionid interacts with tomcat which keeps upon that sessionid the application object in memory with all the operation done there until now. In case 10 tenants call each that tomcat via resp. the own browser, tomcat will create 10 application objects, and give to the tenant (the user) his/her sessionid. Each interaction is done using the sessionid (standard tomcat) and reaches only the associated application on tomcat side.

I do not know yet what a boilerplate is (besides in my kitchen) - and will search for its description. SWISH seems to give at each predicate call all the environment (prolog kb + goal) in a JSON String to the server. This costs for the prolog engine the loading of all the prolog code before the execution. In a scenario like I depicted above, the loading is already done and “personal” to each user or tenant.

Is there a way to implement this in a mature fashion with swi-prolog ?
Swi-prolog has a very nice web service, is this web-service multi-tenant ?
Or should it run under a tomcat (is there a .war way to use swei-prolog ?)

Thanks

#4

The terminology in SWI-Prolog is rather different. If you want to provide session aware services you have several options:

  • Use library(http/http_session), which lets you create a session and store data that is specific to this session. Otherwise there is no state and you must be aware that different requests from the session are typically handled by different threads. The session data is not persistent (disappears on server restart). The latter isn’t that hard to add, but is has never been done.

  • Use a Pengine. You use a JavaScript call to create such a beast. After that you can ask it as many queries as you like. The Pengine is a thread with a temporary module for its data. A Pengine is associated with an application that can provide as much shared program code as you like. The client can also inject Prolog code into the Pengine to let it do server side computation. This is not bound by a session: the JavaScript code has to remember the Pengine’s id, which limits this to a single (dynamic) HTML page.

  • You can also implement something similar to Pengines yourself, based on threads or engines: if a session is opened (there is a callback for that), create an engine that represents the session. Any request on that session that needs to access the state uses engine_post/3 to do the stateful computation. This is not hard to get going. Engines are fairly cheap, especially if you call garbage_collect/0 just before waiting for the next interaction. I could provide a very small demo illustrating the idea.

#5

Thank you very much for your hints/advice. I will study them and see how to procede (I seem to be the first having this wish).
Yes please, in case you could provide some examples - thank you in advance (here or private).

#6

I’ll see whether I can find some time. In the mean while though, it isn’t really clear what you want to preserve for which reason during a session. The more complicated Pengine or engine approach allow you to use the Prolog flow of control as part of the state. The normal session data merely provides a Prolog dynamic database that is associated to the session.

#7

In order to be quick and secure on web service execution, I would like to know that my application server (e.g.) be hosting a prolog interpreter with loaded kb’s for each web user individually (what would be the cost here). So that there is no need to re-load any kb from scratch when that particular user executes some goals. In general this is how tomcat would handle that.

  1. new user on JSP servlet (e.g. JSP but this can be any tech) request -> generate sessionid and allocate application (e.g. prolog listening) in memory
  2. on user actions (goals), execute goals (without reloading any kb each time) and respond to user.
  3. on user logout, release application, invalidate session.

There should be no need to interactively backtrack and do what a prolog user does/did since years inside a prolog console. This would be a use of prolog as dedidated interpreter executing a task, accepting some interactions via input/ouput not necessarily backtracking - with a user env - each user another env. Is this today possible? There is a need to encapsulate execution and env to isolate every (web) user from the other and to avoid reloading of kbs on goals (*), to preserve a state and an interaction via input/output.

(*) of course not talking of lazy evaluation, which is a sane principle to load if needed, to compute if needed.

#8

I’m not entirely sure this is what you are after, but please have a look at https://gist.github.com/JanWielemaker/9b0661fa23a5298ba600428b2389dfa9

This uses the engine approach to create a private space. It also uses a private module. This isn’t really needed: you can also define the predicates that represent the user’s knowledge bases a :- thread_local p/1, q/2, ..... In most cases the latter approach is probably easier. Only of the different users have knowledge bases that are completely differenty structured (using different predicates that are not known a priory), the temporary module approach becomes necessary (that is why it is used for Pengines, where every user writes his/her own program (=knowledge base).

Does this start to make sense?

#9

This is de-facto interesting in general and represents one of the major steps of a prolog env to get industrial. Thank you very much for your link and example code. On my website services start by a JSP page (I can change that). After that, a session is already computed by Java.

What I cannot (yet) figure out is the multi-tenancy on the prolog spaces. The following is how I still understand it from your code and proposal (thank you): I start on a linux server an swi-prolog process inside a terminal and load what you show in the link. Then the next user calls the same web address served by that special prolog web service predicates (as per your link). So in this way we keep ONE prolog space (engine?) for several users.

How can the multi-tenancy here be preserved (a tenant being a company, inside which several users might work and (if we want…) share some prolog space inside a tenant.) ?

A possible generalized scenario could be:

User 1 calls the served address at time t1 and executes some goals (which might affect his kb)

User 2 calls the served address at time t2 and executes some goals (which might affect his kb)

.

.

.

User m calls the served address at time tn and executes some goals (which might affect his kb)

I need to preserve for each user a completely secured prolog space, forbidding any side effects e.g. User1 might perform (assert/retract/load/drop) on the kb, so that User 3 suddenly gets another knowledge base … ?

Is this something your nice prolog server predicates could handle?

Thank you

#10

Yes in the sense that isolation is preserved if the code executed by the server is not malicious. This implies that the code must keep its state in thread_local predicates or in the private module and should not modify anything else. If tenants can upload code and cannot be trusted this breaks down.

The Pengine approach can deal with malicious code, but now the code must be injected by the client while the server provider can provide additional code that can be reused by all clients. The Pengine concepts for web programming are rather different from anything else found in other environments.

If you want a safe approach, forking a new process for every session is probably the best way. Now, SWI-Prolog cannot fork if the process has multiple threads (because we cannot guarantee the state of mutexes). What we can do is start a second single threaded Prolog process that preloads all shared code and listens using a pipe to the main web server. If the web server creates a session it asks for the second process to fork itself and subsequently it talks to the new forked process in a similar way as the communication to the engine works in my gist example (except that we use a pipe to communicate)

I don’t think that is very hard to implement. The main disadvantage is that its scalability is much worse because a process requires a lot more resources than an engine. If a tenant is a company and companies do not trust each other, but they write the code that handles multiple users we can opt for a hybrid approach were we fork a process for each tenant and create engines inside this process for each user. Now, if the tenant makes sure its code adheres to the isolation requirements all is fine. If a tenant doesn’t adhere to these standards its users are not isolated, but they remain isolated from the users of other tenants.

From a web development perspective this is all pretty much similar as the gist, except that the some code may need to be in the main web server and some in the forked processes. That can be achieved using a suitable file structure, but we can be flexible as migrating Prolog code from one process to another is pretty much trivial.

#11

It seems indeed that forking be the best and safest alternative.
Now, from a tomcat/Java point of view, tomcat has already the possibility to fork a process and to call inside that process an application (which chould be a lean swi-prolog application) - or to instantiate a “big” swi-prolog class having all inside, the interpreter … So a tomcat servlet could control this forking.

Do you think I could encapsulate swi-prolog in that way? Could you theoretically / practically provide (maybe there is already) a library from which the “big swi-prolog class” can be instantiated and used (right now I forgot in which language swi-prolog is realised, if not in Java, then shell programming will help (me)). Of course I do not intend to cause (you) work on this, unles this could be generally interesting.

Term Expansion issue when consult is called from a Module (was: How to get "Consult('Test.pl'), predicateFromInsideTest." to run)
#12

See https://gist.github.com/JanWielemaker/d9837fb1858d76d5dfd9caa3851db057 for an example fork based way to manage Prolog compute servers. You can load all common code along with the fork server, to make a servlet available quickly.

This assumes a Prolog client (included). If you want a Java client I’d rewrite the server to use JSON for exchanging queries and replies. There are zillions of things that can be added to this setup (resource limitations, non-deterministic answers, I/O forwarding, etc.)

#13

Impressive - thank you a lot for this code. The code is prolog-centric and needs at least one prolog instance running. What if that instance breaks down?

( Of course I would prefer a Java code called and controlled by tomcat (using tomcat security) … I will study the use of this code however. )

#14

Yes, you need one running. It barely does anything though, so it should be very reliable. Normally you’d start that under systemd or some other service manager and tell that to restart the service on failure.

Doing the fork in Prolog allows you do preload (large) background knowledge and anyway is a lot faster than starting a Prolog instance fresh and typically uses less resources as the multiple instances will share a lot of read-only data/program code.

#15

OK

It barely does anything though, so it should be very reliable.

well … for me reliability means other things. Anyway - supposed the servlet talks to that prolog instance which forks each time a process - in one forked thread what can be done please? Can the prolog kb be changed? If one user x change the kb in “his” forked prolog thread, the others still have an unchanged and (from x) unreachable prolog system with bound input/output they can change at their side? Tomcat will take care of the network security, but this one running prolog … is this the place where every forked process communicate or is it just to fork new ones? So then the servlet will have to talk just with a forked prolog thread (not with the common prolog one …) right ?

#16

Hello Jan

  • a happy new year -
    Now that (my) lecturing is complete (for 2019) I have some time to dedicate to your nice prolog and to my issues of integrating it into a java tomcat 9. Re-reading what I wrote you maybe I guess I could not make me fully understood.

What I need (and would like to do) is the following secured prolog encapsulation:

Inside my tomcat java web application I would like to start upon some client buttons via javascript calls (but always at server side!) a tenant (session bound) instance of prolog in a forked process, in which the java code is putting commands and readings reactions (I/O) and giving these reactions back to the same session client. Think of a (tenant) private prolog shell listening to load and execute commands which come from an attached java program. I discovered JPL just a minute ago, donloaded the code (thank you) but I cannot find there a real example where a Java program starts / forks a private prolog process (in a kind of shell), then load some scripts and query some clauses. And I could not find the name of the jpl jar to bind with my java code. Maybe I am blind, any help would be appreciated.

Cheers

Fabio

#17

I could find on my computer searching for “jpl.jar" a couple of files with that name. Then I tried to download jpl.jar from http://www.java2s.com/Code/Jar/j/Downloadjpljar.htm in any case I have troubles binding that jar with the rest of my java.
Normally I have no problem adding and using a jar for my projects - I simply add the jar to the WEB-INF/lib folder of the application and code in Java. With jpl.jar it seems it does not want to function as espected or assumed. For instance my eclipse wants to rename Query into jpl.Query and then tries to import the code from org.jpl7.Query - but at runtime it does not function (it does not find the class or jpl.fli.Prolog cannot be initialized) … Seems to be difficult … until it wants to run… Maybe I am using an old packaged jpl.jar - which one is the good one? (anyway In my java projects I have jpl.jar once! (so no ambiguity problems)).

The code:

public class PrologEngine {

**public** String create() **throws** IOException

{

	String xoutput = "PrologEngine starting ...";

	// create via fork a prolog process which never ends

	jpl.Query q1 =

		    **new** jpl.Query(

		        "consult",

		        **new** jpl.Term[] {**new** jpl.Atom(“/…………/test.pl")} 

		    );

	jpl.Term x = q1.goal();

    **return** xoutput = x.toString();

}

}

In the following the exception (maybe you understand what can be wrong):

SEVERE: Servlet.service() for servlet [ACTIONS] in context with path [/XXX] threw exception [Servlet execution threw an exception] with root cause

java.lang.NoClassDefFoundError: Could not initialize class jpl.fli.Prolog

at jpl.Query.open(Query.java:286)

at jpl.Util.textToTerm(Util.java:162)

at jpl.Query.Query1(Query.java:183)

at jpl.Query.<init>(Query.java:176)

at com.s2m.classes.PrologEngine.create(PrologEngine.java:19)

at com.s2m.classes.ACTIONS.doGet(ACTIONS.java:253)

at com.s2m.classes.ACTIONS.doPost(ACTIONS.java:57)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)...
#18

I set PATH to
export PATH=/Applications/SWI-Prolog.app/Contents/MacOS/swipl/bin:$PATH

where the swipl app is.

Then I deleted jpl.jar from the WEB-INF/lib folder and added the swipl original external jar directly using eclipse’s java build path - “/Applications/SWI-Prolog.app/Contents/swipl/lib/jpl.jar” and the jar loaded. Again the not-found issue. This time getting a java.lang.ClassNotFoundException: org.jpl7.Query

Looks more like a chess party :wink:

Chers

F

#19

Finally I got the solution, fixing paths after paths:

java.lang.UnsupportedClassVersionError: org/jpl7/Query has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

This corresponds to Java 11.

I am using Java 8.

What do you suggest please? Can I have a version for java 8 please ?

#20

Even using jdk 11 and after having entered the missing java argument -Djava.library.path=/Applications/SWI-Prolog.app/Contents/swipl/lib/x86_64-darwin - I stuck in
Jan 03, 2020 12:43:35 AM org.apache.catalina.core.StandardWrapperValve invoke

SEVERE: Servlet.service() for servlet [ACTIONS] in context with path [/SVIZ] threw exception [Servlet execution threw an exception] with root cause

java.lang.UnsatisfiedLinkError: /Applications/SWI-Prolog.app/Contents/swipl/lib/x86_64-darwin/libjpl.dylib: dlopen(/Applications/SWI-Prolog.app/Contents/swipl/lib/x86_64-darwin/libjpl.dylib, 1): Library not loaded: @rpath/libswipl.8.dylib

Referenced from: /Applications/SWI-Prolog.app/Contents/swipl/lib/x86_64-darwin/libjpl.dylib

Reason: image not found

Using SWI-Prolog (threaded, 64 bits, version 8.0.2) on mojave 10.14.6