File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Java in General and the fly likes how to join data in java without using Database Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "how to join data in java without using Database" Watch "how to join data in java without using Database" New topic
Author

how to join data in java without using Database

Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 639
I have to join data in java(but i cant use database).I have data coming from 3 sources (just like it comes from 3 tables but not database tables).Had it been from SQL tables I wud have applied sql join and joined them.But this data is not coming from SQL but from text files.Now I am confused how to join them.Please suggest me the way.One way I am thinking of is putting these in list and then iterating one outside other and matching but that would be complex.Please help.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

It's not clear exactly what you mean or just what format these text files will have. I'm assuming it's something like this:



And you need to join them into


If so, I suggest:

1) Define a class corresponding to the joined row.

2) Define classes corresponding to each of the "sub-rows" in the individual files.

3) Create one SortedMap per file. The keys will be the IDs or whatever the common column is that you're joining on. The values will be the objects corresponding to that file's "sub-rows".

4) Read each file, populating the appropriate Map as you go.

5) Pick a Map as your starting point. Iterate over its EntrySet(). Create a new "joined row" object for each entry, filling in whatever data is present in the value of that map. Call get(key) on each of the other Maps to get their corresponding data and fill in the rest of that joined row.
Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 639
yes exactly this is what i want do do.Can this be done in a simpler way using ArrayLIsts?
Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 639
One way I am thinking of is putting these in list and then iterating one outside other and matching but dont know the complexity of it.
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11230
    
  16

part of how to solve it also depends on the scale of the file size. If each file only has a few hundred rows, just about anything will work.

if you are talking about millions of rows per file...how long do you want to wait for it to finish?

Software design is always about balancing many competing factors. You have to figure out which are the most important, and then let the others suffer slightly. Whether it is memory, complexity, speed, technologies (i.e. database vs. arrays) or something else...you can never have it all.

So CAN you put them into lists and iterate? Yes. but you will sacrifice speed and efficiency.


There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors
Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 639
no worries for speed.because I am using all this logic in Hadoop ..only thing I am not clear about is once i get all this data in 3 lists..how to join comparing and joining..logic will become quite complex.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

The maps and classes I described will not only be faster than lists, the code will also be cleaner and simpler to understand.
Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 639
these files are picked by map reduce algorithm of hadoop.

thanks all..will try the logic
 
Don't get me started about those stupid light bulbs.
 
subject: how to join data in java without using Database