Connection with an external user database

[Migrated From Jforum.net]

I want to create a lightweight connection between JForum and our own database. It seems like a good idea to make this connection as generic as possible so others might benefit from this work.

Before starting the implementation I want to discuss my approach with you guys so you can help me spot problems early or suggest improvements.

I hope you can follow my explanation, if not I will try to elaborate as best as possible!

Basic Idea
----------
I propose a 'lazy' connection between the JForum user database and the external system. Each time a user logs in, JForum will query the external system and update its knowledge of the user.

Implementation
--------------
- Create an interface "PassiveLoginSystem" that contains two methods:

        public int validateLogin(String name, String password);
        public String getName(int userId);
        public int[] getGroups(int userId);

(validateLogin returns the userId of the user. if your external system does not use numbers, your PassiveLoginSystem implementation needs to map these to numbers, possibly using an extra database table).
JForum will contain a single object that implements this interface representing the connection with the external system. In most cases, this interface can be implemented simple enough, for instance using queries to a database.

*** Note that after we have got this working, implementing this interface is the only thing needed to connect JForum to a new system!

- Disable the JForum "account creation" functionality

- Disable the JForum "name change" functionality (if there is any?)

- Disable the JForum "users in group" editing funtionality

- Extend the UserVH.validateLogin() method with the following logic:
1 Get the userId of the user by calling the PassiveLoginSystem.validateLogin method (with the name and password taken from the HTTP request)
2 Retrieve the name of the user by calling PassiveLoginSystem.getName
3 Retrieve all the JForum "User" information using UserModel.selectById
4 If the user does not exist yet, create a new JForum account
6 If User.username or User.password differ from the current name, update these fields and call UserModel.update to store the changes in the database
7 Retrieve the groupIds the user should be part of using PassiveLoginSystem.getGroups, remove any groupIds that are currently not valid JForum groupIds
8 If the groups do not match with the current groups the user is in, use UserModel.addToGroup and UserModel.removeFromGroup to make things right

I'm also contemplating an extension for systems (like ours) that only allow
unique usernames:
5a Find all users with the same name (but a different userId) using UserModel.findByName(name, true)
5b For each such user, request the current name using PassiveLoginSystem.getName(userId)
- if the user is present, update his/her name must have changed so change the name in the JForum database to match the new name
- if the user is no longer present (getName returns null), place some extension behind the current name, like "*deleted*" or something to signal that the user does no longer exist.

This approach could be configured by adding some simple configuration options, namely the fully qualified classname of the PassiveLoginSystem implementation to use, and a boolean option whether names should be unique.

Rafael, there are some things I'm not quite sure about:

- It is not yet clear to me how this approach might interact with the "autoLogin" feature, can you elaborate on that? Is the UserVH.validateLogin method actually called when someone logs in automatically?

- The roles etc. can be managed by the admin as usual, provided that the admin user is "bootstrapped" during installation? (I guess some user from the external system should login once to create the account and should then be assigned "admin" rights in JForum by hand in the database?)

Of course, I'm very interested in any other remarks!

[originally posted on jforum.net by Pieter]

[Migrated From Jforum.net]

I read your post and I am thinking about it. 'Will reply soon.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Hello Rafael, have you had time yet to reflect on this approach? I will need to finish a working implementation in about a month, so time is starting to become an issue.

Regards,

Pieter.

[originally posted on jforum.net by Pieter]

[Migrated From Jforum.net]

Pieter wrote:Hello Rafael, have you had time yet to reflect on this approach? I will need to finish a working implementation in about a month, so time is starting to become an issue.

Well pieter, your approach is one on many I have heard / considered in the last days, and all of them fit into a particular situation. As you suggestions about changing some piece of code may help in some way, they fail on others, and, sometime, it will be necessary to change the jforum's root code here or there.

For instance, my suggestion is, if you have the need to integrate jforum with a particular system, do it according to your needs, if if that wasen't "The One"... Maybe with your experiencie integrating the systems and the ones I have done, we can elaborate a good login and auth system.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

That sounds reasonable.

However, I don't like the idea of having to apply my own patches (and possibly debug the result) each time I upgrade JForum. Granted, new versions could still introduce bugs specific to my situation, but I believe these will be less likely when other developers can actually see the changes I made.

So do you mind if I commit my changes in the main source tree?
I will adhere to the following rules:
- I will signal my changes by clear comments to facilitate a possible rollback in a later stage
- I will make all changes depend on configuration option(s) so the semantics of the "normal" situation are left completely intact.
- I will try to implement most changes in separate classes/interfaces to avoid clutter in the main source tree.



[originally posted on jforum.net by Pieter]

[Migrated From Jforum.net]

Hi Pieter,

Can you be more clear what you mean by an external system?

I am guessing that you wanted a user to log in elsewhere and be able to access the forum. Am I right?

Thanks and Regards,
James Yong
[originally posted on jforum.net by jamesyong]

[Migrated From Jforum.net]

That is correct. We have developed a multiplayer game platform (http://www.gamesquare.nl).
We currently have a user community of over 30000 active players. We are going to introduce a membership system in a couple of months, and we want to offer our members a forum.

We want to integrate this forum seamlessly with the rest of the system, so we want to use our player/member database for forum authentication.

If a player is logged in our game platform, it should only require the push of a button to enter the forum without the need to enter a name or password again.

Members will be able to change their name in our user database and the forum should pick up these name changes transparently.

We have several levels of users, and each level should have their own set of forum boards they can access.

When members loose their membership, forum access should be denied as well.

[originally posted on jforum.net by Pieter]

[Migrated From Jforum.net]

Pieter wrote:
So do you mind if I commit my changes in the main source tree?
I will adhere to the following rules:
- I will signal my changes by clear comments to facilitate a possible rollback in a later stage
- I will make all changes depend on configuration option(s) so the semantics of the "normal" situation are left completely intact.
- I will try to implement most changes in separate classes/interfaces to avoid clutter in the main source tree.

Sure, you can commit your changes to cvs. Your suggestions in the first post were nice, and would help someway, so I'm trusting on your work

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Pieter wrote:
If a player is logged in our game platform, it should only require the push of a button to enter the forum without the need to enter a name or password again.

Members will be able to change their name in our user database and the forum should pick up these name changes transparently.

I supported a company to integrate jforum - using pgsql - with their system. One of the modification was that they already had an users table and need to use it. So, we "merged" both tables ( the one they have, and jforum_users ), and it worked very well, since the modifications were really little.

Another point was that they didn't want to use the jfourm's login system, but they own. This one resulted on a more elaborated patch, because, in order to use jforum's DataAccessDriver, it is necessary to have the ThreadLocal data ( connection, request and response ) correctly set. So, at JForum.java, we put a public method named configureThreadLocal(Conneciton, HttpServletRequest, HttpServletResponse), so then they use this method *before* accessing jforum's stuff. Probably I'll put this method in the cvs too.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Rafael Steil wrote:

Another point was that they didn't want to use the jfourm's login system, but they own. This one resulted on a more elaborated patch, because, in order to use jforum's DataAccessDriver, it is necessary to have the ThreadLocal data ( connection, request and response ) correctly set. So, at JForum.java, we put a public method named configureThreadLocal(Conneciton, HttpServletRequest, HttpServletResponse), so then they use this method *before* accessing jforum's stuff. Probably I'll put this method in the cvs too.

Rafael

Hmm, this feature seems great to add for jforum
[originally posted on jforum.net by jamesyong]

[Migrated From Jforum.net]

It's something like this: https://coderanch.com/t/574564

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Thanks
[originally posted on jforum.net by jamesyong]

[Migrated From Jforum.net]

The basic implementation turned out to be much easier than expected. I have written a special UserModel database driver that implements all the required extra login logic in validateLogin, and then dispatches to a delegate UserModel (for instance mysql.UserModel) to finish the validation process. All other methods in the UserModel are dispatched directly to the delegate UserModel.

I still have some way to go in polishing my code and adding some related functionality (for instance supporting disabling of password changes which make no sense in my case) and I need to test some more, but things are looking good. When I'm finished I'll write step-by-step explanation of how to connect JForum to an external user database.

Basically, you need to write three sql statements and fiddle some configuration options and your done :P


[originally posted on jforum.net by Pieter]

[Migrated From Jforum.net]

I am considering using JForum and need this functionality (use my own DB tables and logging mechanism to authenticate users). Is this functionality in RC5?

Thanks,
Otis

[originally posted on jforum.net by Anonymous]

[Migrated From Jforum.net]

Well, Pieter added some stuff and he says that works partially, but i did't checked the code yet. You can take a look at net.jforum.view.forum.UserAction, method "validateLogin()".. At WEB-INF/config/SystemGloals.properties there is also a key named loginMode.
You can take a look in this pieces of code.

Sorry not helping much in this point :roll:

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

I would like to use JForum, but everything related to user login and administration needs to be separate.

It would make sense to separate out features such as forgotten password, login, user to group resolution etc. into completely separate packages. Why didn't JForum rely upon the servlet containers authorization mechanisms? It would make JForum much more usefull as an add on option to existing sites.
[originally posted on jforum.net by Anonymous]

[Migrated From Jforum.net]

Anonymous wrote: Why didn't JForum rely upon the servlet containers authorization mechanisms? It would make JForum much more usefull as an add on option to existing sites.

It is an option, like one hundred other available ;)

Yes, JForum can be refactored to allow this kind of authorization too. But we need help with coding - eg, we're open to new ideas, solutions and improvements, but many times external help is needed, like in this case.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Pieter,

Do you have a single sign on? I am working on it to but I don't get it to work.

Please let me know,

Henk
[originally posted on jforum.net by henkie]

[Migrated From Jforum.net]

I work for a company that has a very extensible user/authentication model. It works well, we can plug into anything (LDAP is supported out of the box). Here is how it works at a high level:

There are two main classes responsible for all this:

SessionManager
This class is responsible for all session stuff: Determining if a user has a session, creating sessions, deleting sessions, etc.

UserManager
This is the wrapper on the user interface. It implements things like findUser, searchForUsers, findGroup, etc.

There is a SessionManagerBase and a UserManagerBase already implemented which works with our regular user model where everything is in the DB. There is also LDAPSessionManager and LDAPUserManager which extend these two to provide LDAP integration.

To customize any aspect of the users or sessions you simply extend the Base class, and plug in your class to the application. The whole app users the SessionManager and UserManager classes to do everything (well all user stuff). All these classes do is load the configured instance of SessionManagerBase and UserManagerBase and call the corresponding method on the base class.

So for example. Lets say you want to implement your own createSession function. You would create a class FooSessionManager that extents SessionManagerBase, and just implement createSession(request, response). The you just tell the app to use FooSessionManager instead of SessionManagerBase.

When the core code in the app is trying to create a session, it will call SessionManager.createSession, which will call FooSessionManager.createSession.

It works well. Let me know and it this is helpful and I can provide more detail.

The main goal is to allow other people full control of the user and session management without forcing them to deal with the core code.

[originally posted on jforum.net by Anonymous]

[Migrated From Jforum.net]

Hi, Pieter,

I saw your message at jforum.net. I also need to connect JForum with our own database system. Looks like you did it sucessfully. You said you would write step-to-step instructions in implementing this, can I know where is your message, or you never get a chance to write it,

Your help is highly appreciated!

Sharon
[originally posted on jforum.net by gyang]

[Migrated From Jforum.net]

gyang, is there a reason you are not considering SSO? Or the posting I made about a "weak coupling" of JForum and my webapp using the JForum cookies returned by a login http request? I think both of these solutions are better than any solution that involves patching the code.
[originally posted on jforum.net by time]

[Migrated From Jforum.net]

I had hard time to configure the database. In our own application, we use file system to store everything. We don't use mysql or any available database at all.

So when I configured the database type in WEB-INF/config/SystemGlobals.properties, in the following content:

#####################
# DATABASE SETTINGS
#####################
# Database type to use
database.driver.name = mysql

What should I choose? Just name whatever name our file system is? Or the external database has to be sql-related, since lots of code are binded with java.sql, such usersession, simpleconnection, or poolconnection.

Thanks

[originally posted on jforum.net by gyang]

[Migrated From Jforum.net]

You should have a database to use JForum. Either mysql, postgresql, oracle or hsqldb (in memory database) are supported.

Rafael
[originally posted on jforum.net by Rafael Steil]

[Migrated From Jforum.net]

Hi, Rafael,

Thanks for the prompt response. I am actually writing a database driver to connect JForum with our application.

When I configure the database using our own driver ( I added this driver in Login.html, put our jar file in webapps\JForum-2.1.5\WEB-INF\lib). I got the following error:

Expression dateTimeFormat is undefined on line 2, column 29 in default/header.htm. The problematic instruction: ---------- ==> setting datetime_format set to "dateTimeFormat" [on line 2, column 1 in default/header.htm] [on line 2, column 1 in default/header.htm] in include "header.htm" [on line 1, column 1 in default/message.htm] in include "${templateName}/message.htm" [on line 5, column 1 in exception.html] ---------- Java backtrace for programmers: ---------- freemarker.core.InvalidReferenceException: Expression dateTimeFormat is undefined on line 2, column 29 in default/header.htm. at freemarker.core.TemplateObject.assertNonNull(TemplateObject.java:124) at freemarker.core.Expression.getStringValue(Expression.java:118) at freemarker.core.Expression.getStringValue(Expression.java:93) at freemarker.core.PropertySetting.accept(PropertySetting.java:110) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.MixedContent.accept(MixedContent.java:92) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.Environment.include(Environment.java:1375) at freemarker.core.Include.accept(Include.java:155) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.MixedContent.accept(MixedContent.java:92) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.Environment.include(Environment.java:1375) at freemarker.core.Include.accept(Include.java:155) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.MixedContent.accept(MixedContent.java:92) at freemarker.core.Environment.visit(Environment.java:196) at freemarker.core.Environment.process(Environment.java:176) at freemarker.template.Template.process(Template.java:231) at net.jforum.exceptions.ExceptionWriter.handleExceptionData(ExceptionWriter.java:108) at net.jforum.InstallServlet.service(InstallServlet.java:143) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at net.jforum.util.legacy.clickstream.ClickstreamFilter.doFilter(ClickstreamFilter.java:59) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:667) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595)

Any hints why this is happening?

Thanks and truly appreciate your time!

Sharon
[originally posted on jforum.net by Anonymous]

[Migrated From Jforum.net]

forget to login in. The above message is from me. Sorry, ops:
[originally posted on jforum.net by gyang]

[Migrated From Jforum.net]

This error is just the template breaking.. you should look in the server logs for the real errors - for example, a "ClassNotFoundError", or anything similar

Rafael
[originally posted on jforum.net by Rafael Steil]