Hello everyone. I started learning Java (and programming in general) only recently and thus I don't know how to do much so please do go easy on me . Right, I'll try to explain what I'm trying to do:
I'm working with a browser-based CRUD application (well it's a little bit more advanced than that but yeah..). In general I want to make an application that can connect either to the crud app or through it (using the functionality) to connect to the server. The Long-term goal being to add a layer of abstraction allowing us to do lots of neat stuff (such as get more complicated statistics, use pre-set templates to avoid common errors and simplify work and so on..). It's a wonderful idea, but since I'm pretty much clueless about how to pull it off at this point it's going to take some time (which is OK since the goal is to learn and if I should incidentally simplify my work in the process that would be a nice perk). However I'm pretty much stuck on how it is you actually connect to and manipulate a web-based application(or a website for that matter)...
My main issue is how to connect and exchange information with the application? More specifically for my smaller projects I need to be able to do the following:
1.) Manipulate the elements on an already active screen...which is just a fancy way of me saying I want to feed the program a list of abbreviations (such as RU, BG and so on), have it check for a match in the open browser window and if one is found to check the relevant check box on the active screen.
- the simplest way of realizing it I came up with is to simulate keystrokes on the keyboard, since this can be done manually with the following sequence: *copy abbreviation* -> press ctrl+f on the active screen and paste it in-> search for it-> press esc-> press tab-> press space. Since this process needs to be repeated more than 30 times it's quite cumbersome to do it by hand. The problem here is I have no idea how to do that and where to look for a solution so any hints are welcome
- the other way is to manipulate the check boxes by somehow referencing them except I don't know how and if I can refer to objects on an active webpage. Again any hints as to how to do that or books, or general direction I should be looking in would be greatly appreciated.
2.)Programatically navigate to,refresh and draw data from a screen without making it visible(but it can be open in the background). The screen being a table with data which, for some weird reason, has a set limit of maximum items it can display. The issue being I need more than the maximum limit for a statistic so I want to refresh and get the information over a set period of time and have my application reprocess it and generate a report I can use. How I'm currently accomplishing that is by periodically copying the information manually and pasting it into an Excel VBA sub I wrote which processes it further and prints out a report.
I basically want to make the manual part of that automatic. Once again, any suggestions are appreciated
The main questions are:
*) How does the interaction between a Java desktop application and a web application work in general?
*) How to log in using Java?
*) How to navigate the app programatically?
*) How to refer to and manipulate the elements?
*) How to emulate keystrokes with Java?
*) How to request and receive information from the web application?
Thank you for taking the time to read through my topic and any guidance you provide.
The best solution I see is have your desktop app connect to the server directly, tell it what needs to happen, and then have the server propagate the changes to any web client with an active connection, using something like WebSocket.
This way, you have one API for doing stuff on the server, and the desktop app doesn't have to care about the existence of the web app at all. As a matter of fact, you can even have different web apps notify eachother of changes this way.
Thank you for taking the time to give me some suggestions.
I will look into web services as a solution later on when I know a bit more about web programming with Java. From what I read through it might not be what I'm looking for since, from what I understood, it keeps an open channel between the server and the app and keeps the client up-to-date with changes. I'm not looking to have a constant connection with the server. I just need to periodically (for example: every 1 to 5 minutes) establish a connection and withdraw some information (if I can connect to the server directly I will need to do this only once since the limitation on the data is set in the web app).
What I'm looking to do is more of using the web app as a checkpoint between the desktop app and the server. Why would I want to do that? I don't want to skip the web app since it already offers all the basic limitations and functionality I'm looking for. If I skip it I'll have to reinvent the wheel by rewriting all the java script code as java code (or just copy-paste it and run it like it is)... My point being whatever I write will be untested and could potentially do more harm than good. Instead I want to make an app that just adds on the functionality the web app offers (more layout options, better filters,templates etc.). And I want it to be a desktop app so I can add a local buffer for some items since we have system down issues due to a bad connection.
What I need to find out in order to do that is a way for the two applications to exchange data. For example the web app places a restriction on the log for the processed items, which can show only the last 200 items. Since we need to see more than this the solutions I could think of were:
1) to periodically check and copy the table into an excel sheet and have a vba app process the whole thing at the end of the day so we can see all the unique translations for the day - this works but it's mostly done manually and is not a good solution
2) to make a program that programatically does this every x minutes, where x is a user-defined value, and doesn't disturb the user - that's what I'm currently trying to do.
3) make a program that gets the whole list for the day from the server - best solution, however since I have no knowledge of the server I can't be sure that doing this won't affect it in some negative way...
So generally I'm just looking for a way to reference/manipulate elements from the web-app http/java script code... Simply put: If I have a web page http://www.testpage.com with a text field with ID:name and a button with ID:submit. How would I give the text field a value of "Angel" and submit it? Would I go about it like programming for Android, create a variable with the object reference ( TextField nameField = (TextField) findViewById (address.id.name); ) or would I bypass the gui controls and directly trigger an event by using a GET/POST http request? (This is totally made up for the purposes of illustrating what it is I'm trying to accomplish)
Otherwise I found a solution for one of my current problems - emulating keystrokes. I stumbled across the Robot class (java.awt.Robot) while skipping through the documentation. It looks like it will work for what I'm trying to do (check some check boxes on the web app programatically). Granted, it's not an elegant solution but it works and I can implement it with my current knowledge ^___^; ( I hope).
Thanks again for the suggestions I will look into them in more detail when I know enough about web-programming to understand how they work
Stephan van Hulst
Joined: Sep 20, 2010
Angel, to be honest, all of these sound like really horrible alternatives to a much cleaner, easier and robust solution. The only problem is that you think it's boring: Rewrite the desktop app in Java and test it properly.
By the way, I'm curious: What functionality would you have to repeat in your webapp? A lot of functionality can be on the server.
thank you for answering my question. As I mentioned from the start I'm pretty new and don't understand how things are done yet. So naturally my first thought was "oh, but can't I just press the buttons programatically..." so until you said it's too hard and buggy to be worth the bother I didn't understand. I never said the solution you offered was boring, sorry if it sounded that way, what I meant was I don't really understand how to make it work at this point. Also I have no knowledge about the server. And don't know how to get the info needed to connect to it directly.
As for the functionality, I haven't spent a lot of time analyzing the java script code (since it's nearly uninteligable) so you might be right about that as well.
As I understand I should look at client-server architecture and web services in order to make the application.
Nah, I actually like testing, in fact I'm holding off on writing until I can learn how to do TDD.
So yeah it's an already existing, fully-functional application. Let me start from the beginning. I work as a coder for a company doing marketing research. Basically we use a web application to communicate with the database server, the protocol is https, the server is in another country and we are only vaguely aware of its existence. Since I'm starting to learn programming officially from next week and I observed several issues that could be solved and make our work and life significantly easier I decided to try to solve them as practical projects (so that I have additional motivation to learn properly and maybe make something practically useful). Anyway most of those issues are due to the fact that the requirements changed but the system wasn't updated and cannot be tailored to our needs since it's one system for the whole world. That's when I thought to research the possibility of making a desktop add-on which would add additional rules, views and allow them to be changed according to what we need. But my main issue is still how to make the two exchange information and thus my post here...
As far as what I understand of the application it's a web-client restricted to run within the corporate network or via citrix emulating the corporate network. It's written in html/css/java script and I suspect the java script code is machine-generated because it's really horrible as a layout, so I have a hard time understanding what's what. In design terms it's pretty much standard - you have a login screen which after validation takes you to the home screen from which you can navigate to whatever it is you need to see(which in most cases is the list of retailers and their respective contents so that you can link the articles to the database). As I said it's a little more advanced CRUD web app. Because I think some of the basic rules and limitations are coded on the client and I'm not sure the server performs a double check for these rules I really want to avoid messing with it directly mainly because doing the wrong thing might cause it to crash and that would be really bad for me . Also if I were to redesign the whole thing I would have to download about...maybe 100+ pages of horrible java script code, analyze it, figure out what it does, figure out how to convert what it does to Java and write it.
To give you a simple example of a thing I'm trying to accomplish: There is a form which gives us information about our output (items linked to the database,translations), however it has a client-side limitation of showing max up to 500 items but recently it has become important for us to keep track of the number of translations we do. That being said some of us do over 1500 translations per day...but we can only get the last 500...so basically we need to manually copy this list of 500 and periodically repeat this and calculate our daily output based on that. To get to this form one would need to perform the following steps:
Log in-> Go to the list of retailers -> Open a retailer -> Press Output button to open the form. We also like to get additional statistics from the output data like, which retailer had how many articles for the different product group (calculating volume), how much time it took to process the retailer ( for feedback and better retailer distribution between the team members responsible for the country). I partially solved the problem by writing a VBA macro which breaks up the total copied data into such statistics. However I would really like to make an application that can do this automatically over the course of the day without requiring the user to do anything apart from starting it. The things I came up with were - to either emulate what the user does manually (like using the Robot class) or to somehow get the information programatically. Obviously the Robot class approach will disturb the user until it finishes so I'm looking for a way to get this information programatically...so far however I haven't made any progress...
If you need anything more specific just ask, this was all I could think of at this point.