wood burning stoves 2.0*
The moose likes Android and the fly likes Data storage and retrieval choices Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Mobile » Android
Bookmark "Data storage and retrieval choices" Watch "Data storage and retrieval choices" New topic
Author

Data storage and retrieval choices

Mike Riesen
Greenhorn

Joined: Apr 02, 2014
Posts: 7
I've been lurking these forums for a while, between here and stackoverflow usually get an answer by just searching, but I've hit a spot where I might need a little hand holding.

I've written an app for creating guitar tab, at the moment a user can work with only one file at a time, which is just a sharedPreferences that can be saved/loaded. I'd like to support multiple saves and I'm not sure what the best approach is. I think I can work my way through by just treating multiple sharedprefs as save files, but I'm not quite sure how I would retrieve and display a list of those for the user to load. I first thought about using an sqlite db and trash the sharedpref idea altogether since I've done a little work with mysql in a php website type environment, but it got a bit daunting when I was just setting up the basics for a db.

So to clarify, I guess the question would be - Is there a particular common or "best" way to store and retrieve persistent user created data? The current method I'm using has the sharedpref storing integer values only.

Apologies if that's a vague question, let me know if you need more info.

Thanks ahead of time,
Mike
Ron McLeod
Ranch Hand

Joined: Feb 12, 2013
Posts: 292
    
    5

Common options are to use internal storage, external storage (SD card), or database.
Mike Riesen
Greenhorn

Joined: Apr 02, 2014
Posts: 7
Thanks for the reply. I've looked over those links in the past, but it hasn't given me much in the way of which direction I should take unless you're saying its a bad idea to use sharedPreferences in the manner that I was talking about.
Ron McLeod
Ranch Hand

Joined: Feb 12, 2013
Posts: 292
    
    5

Mike --

It's difficult to offer any concrete suggestions when there is really no information to work with. Can you elaborate on the data, for example:

  • What is the source of the data?
  • Is it text or binary?
  • How is it structured?
  • How would it be used?
  • What is the volume of data to be saved?
  • If there were multiple data sets, how would the user determine which one to use?
  • Would it ever need to be shared with other applications?
  • Would it ever leave the device?

  • The more details that you can offer, the better the chance of gettting a useful response.

    Ron
    Mike Riesen
    Greenhorn

    Joined: Apr 02, 2014
    Posts: 7

    What is the source of the data?
    Is it text or binary?
    How is it structured?
    How would it be used?
    What is the volume of data to be saved?
    If there were multiple data sets, how would the user determine which one to use?
    Would it ever need to be shared with other applications?
    Would it ever leave the device?


    Sorry about that, I'll try to answer as best I can.

    At this time:
    1. It will not leave the device.
    2. It will not be shared with other applications
    3. One of the problems I'm having is trying to figure out how the user will determine which data set to use (I pictured a dialog with listView populated somehow via getSharedPreferences() if i keep using sharedprefs)
    4. Should be very small file size, but technically there's not preset maximum on the number of values being stored. As it's written now, 6 key value pairs = 1 strum of a guitar, and this is intended to store the values of an entire song.

    Here's some code from the app related to how I'm saving data now.



    I have a Map<Integer, Integer> that I send to Bundle , that map stores a view id as the key and a resource id as the value. when save is selected it moves all that to a sharedpref.

    And this is how I'm loading it, if it's of any use.



    Hope that clarifies a bit, thanks for sticking with me.
    Steve Luke
    Bartender

    Joined: Jan 28, 2003
    Posts: 4165
        
      21

    Mike Riesen wrote:3. One of the problems I'm having is trying to figure out how the user will determine which data set to use (I pictured a dialog with listView populated somehow via getSharedPreferences() if i keep using sharedprefs)

    One mistake you are making is mixing the GUI with the data storage. These are very different and independent things and you shouldn't let one determine or influence the other. I think your real problem isn't so much that you need to figure out how to display the options, but you need to know what options are available (regardless of if they will be displayed, you need to know what exists and how to access them).

    What I would probably do is more to using normal Java java.util.Properties files to store the songs in since they are platform independent and gives you a means to move the songs of device easily (backup to dropbox, develop a web-app that mimics the mobile app functionality, or allow song sharing...) should you decide you want to be able to do that in the future. You would store these files in a specific location (as described earlier) and you would have two choices on how to get a list of them:
    1) Use the normal java.io.File API to locate the list of files in the directory
    2) Create a 'master list' file which you update with each saved file providing a name=path pair, then load all the names into a list to gain access to the files

    4. Should be very small file size, but technically there's not preset maximum on the number of values being stored. As it's written now, 6 key value pairs = 1 strum of a guitar, and this is intended to store the values of an entire song.
    ...
    I have a Map<Integer, Integer> that I send to Bundle , that map stores a view id as the key and a resource id as the value. when save is selected it moves all that to a sharedpref.

    Here you are mixing GUI with data storage again. This one is really a bad idea. Since you are storing view ids in the file. There is no guarantee that view ids are constant from version to version. So a user could make a file, get an update to their application, and then their saved file would no longer exist. Is there any reason you need each of the six numbers to be on its own key? Could you do: Line1 = 1,2,3,4,5,6. Each Strum is a key, and the six positions are stored in a comma separated list which you can easily split? This saves the file from depending on the GUI and being version dependent (and makes the file smaller as a benefit). It works because the lines are fixed size and always numbers (so no fear of comma being a good value, or values missing).

    [edit] I forgot to mention that resource ids are also not constant from version to version either. You should store some other value (a custom identifier) in the save file, then map the custom id to the resource id at runtime:

    So, again, will help your save files be cross-version compatible and have a side benefit of letting them be cross-platform as well. You could also use a fixed UI, instead of having to rebuild the image buttons in code, you could use an XML file...


    Steve
    Mike Riesen
    Greenhorn

    Joined: Apr 02, 2014
    Posts: 7
    You seem to be confirming some of the fears I already had - That I need to do a good bit of rewriting. When I started the app I had almost no working knowledge of java in an android environment and now I have a (comparatively) much better understanding of it.

    Originally there was no option for persistent saving of music, I just extended what I was doing with bundles (for orientation change etc.) and applied it to sharedPreferences. It was intended as a temporary measure until I came up with a good alternative, and that's how I ended up here. I don't have the code in front of me but I'm pretty sure the drawable resource ids are getting pulled at runtime outside of what I previously posted, either that or I've just been really lucky and they've stayed constant for me. I'll look over your suggestions when I can get my code in front of me, just wanted to say thanks in the mean time.

    edit: ok nevermind, I see what you mean. I wasn't thinking about loading music after a new version. I tested for that and had no problems, but I see why there's no guarantee now.
    Mike Riesen
    Greenhorn

    Joined: Apr 02, 2014
    Posts: 7
    Alright, I finally had a chance to look over everything. I'm not sure I completely follow your example, but what your saying pretty much makes sense to me. Possibly with one separate method I could give all those image resources a custom id, or like you say find somehow to further divide gui from data.

    You could also use a fixed UI, instead of having to rebuild the image buttons in code, you could use an XML file...

    The only thing stopping me from doing that is I don't know how to account for the length of a song. if I limit it to 100 beats or even 1000 I think that might become problematic. I'm not particularly a fan of all the dynamic UI creation I have going on either.

    While we're on it, I know that when we use ids like "R.drawable.Position1" that it will return the respective int, is it possible to do the reverse? Maybe I could store the string instead... or maybe that's what you're saying and I'm being dense.

    Again thanks for the ideas so far.
    Steve Luke
    Bartender

    Joined: Jan 28, 2003
    Posts: 4165
        
      21

    Mike Riesen wrote:
    You could also use a fixed UI, instead of having to rebuild the image buttons in code, you could use an XML file...

    The only thing stopping me from doing that is I don't know how to account for the length of a song. if I limit it to 100 beats or even 1000 I think that might become problematic. I'm not particularly a fan of all the dynamic UI creation I have going on either.

    Well, I don't know anything about your design, so it is really impossible for me to make a meaningful suggestion, but I will try anyway

    It sounds like you have a particular thing (a strum you called it before or beat or whatever) and you want to display many of them. Visually, this sounds like something you would put into a dynamic view, such as a scrolling list, a grid, or a series of cards that you 'flip' through. Although these are very different ways of displaying things they share a few things in common:
    1) They display repetitive data. So you can make a single view which can be repeated (either linearly, in a grid, or in cards).
    2) The number of repetitions can grow and shrink.

    This would seem like a perfect use of one of the AdapterViews such as ListView, GridView, StackView, or ViewPager (technically not an AdapterView, but works with an Adapter so close enough). The basic idea is that you have a list of the 'things' (chords, strums, beats, whatever) and you have a single View to display it (in XML or not). You then have a space to organize and display them sequentially (the AdapterView) and an Adapter which bridges the 'thing' to the view. The Android Network's ListView Guide will make more sense I hope. It specifically talks about ListView but the same principal goes for GridView, StackView, and with just a bit of massaging, the ViewPager.

    While we're on it, I know that when we use ids like "R.drawable.Position1" that it will return the respective int, is it possible to do the reverse? Maybe I could store the string instead... or maybe that's what you're saying and I'm being dense.

    Though technically possible to store the String "R.drawable.Position1" to look up the resource ID (I have done it and regret it) it is not a good idea - very hacky and confusing. Remember the name R.drawable.Position1 is just a representation of the element, and so is the int value it holds - just identifiers. It doesn't matter what identifier you use, you just need a way of storing the identification in a file that is flexible enough. You then need to map whatever it is you store in the file to what the UI/Android uses (the Resource ID int).

    So pretend you are using a preference file which stores key value pairs like Java Preferences and Android SharedPreferences as key=value pairs. Your file might look like this:

    See, I am making it up because I have no idea what you are actually storing (maybe if you provide more details I can start describing things in more relevant or meaningful terms). This should be my cue to stop, but I can't help myself sometime... But in my mind, you are storing chords, and a chord is represented by finger positions. You have 5 fingers and six positions. So the above might represent one chord. I have no knowledge of music so I have no clue if that is how chords work, so bear with me...

    Anywho... you need to know how to map "position1" to the drawable you want to display: R.drawable.Position1. So you create a Map in code to do that:


    Then you need to get the bits out of the file. I won't show as much code as I did before, because I made lots of assumptions. For this one, let's just assume you have a method called getFingerPositions() which somehow gets the positions from the file and returns a String[] which contains the five positions as stored in the file in the first bit of code. So you need to translate from those Strings to a drawable resource id that you can then assign to an ImageButton. Here is how it might look:


    So that is the basics.

    In my last post, I also when a few steps further by suggesting you collapse all the finger positions you need for a single chord into one line keyed to chord, so instead of the file looking like what I posted in the first code snippet, the same chord might look like this: Then I made another jump - since the identifiers are arbitrary, you could use simple numbers (ints) instead of Strings to make the file even smaller: . The rest of the code snippet was just about getting the data from a file (I assumed a Java Preferences file) and translating the String you get from the file ("2,3,4,6,6") into an array of ints ([2,3,4,6,6]).
    Mike Riesen
    Greenhorn

    Joined: Apr 02, 2014
    Posts: 7
    The dynamic part of my UI is a horizontalScrollView wrapping a couple of child linearLayouts in such a way that if you "add beat" it appends to the right 6 ImageButtons stacked vertically. Typically this type of guitar tab is written in plain text and would look similar to this:


    e------------------------8------------------------2--------|
    B---------8-10-8-10-8-10---10-8-10----------2--------|
    G----7-10-----------------------------10--7-----2--------|
    D-10-------------------------------------10------2--------|
    A--------------------------------------------------2--------|
    E--------------------------------------------------2--------|


    So obviously the 6 horizontal lines represent the strings of a guitar with their respective tuning to the left. Each number represents where a finger would be on the guitar at a particular point in time (beat).
    It's kind of sloppy, but a "strum" or "chord" is all those 2s at the end there being played at the same time whereas most of the others are being played one at a time on their own beat.

    I don't mean to go off topic with it, but I thought it might draw a better picture for you than posting code. If you go to a website that hosts guitar tab this is basically what you'd see.

    When I first started building my layout I was attempting to use a GridView, I don't recall the reason but I gave it up and adopted what I have now, but that or a group of horizontal (does that exist?) ListViews would be ideal I would think.

    I understand your map now and that sounds perfect, I think I can handle that part now. Reading the map back in sounds pretty straight forward as well.


    In my last post, I also when a few steps further by suggesting you collapse all the finger positions you need for a single chord into one line keyed to chord, so instead of the file looking like what I posted in the first code snippet, the same chord might look like this:


    For someone who says they don't know much about music you seem to have a pretty good grasp. I do have some preset chords built in for use when writing the music, using a somewhat similar method.

    So, assuming I fix the previously mentioned issues... do you think storing the data in a SharedPreferences is the way to go? i think you briefly mentioned CSV as a possibility?

    -Mike

    Edit: apparently the preview uses a monospace font or something because the plain text tab I tried to make looks horrendous after posting it.
    Steve Luke
    Bartender

    Joined: Jan 28, 2003
    Posts: 4165
        
      21

    I am glad you were able to understand what I posted.

    Using a HorizontalScrollView for that might not be ideal. All of the components in the view would need to be loaded at the same time, so it would get memory intensive rather quickly. The AdapterViews only load the visible views and one or two buffers on each end, recycling the views that are no longer on screen to hold the next content - all to preserve memory. You probably dropped GridView because it doesn't scroll horizontally, nor can you get ListView to do so. There are only three choices:

    1) A Gallery: The quintessential horizontal scrolling AdapterView. Unfortunately, it has been deprecated - and I don't know why.
    2) A ViewPager: Also horizontally scrollable, but you only see one 'card' at a time, and need to swipe or otherwise navigate between them.
    3) Roll-you-own or look up someone else's implementation of a HorizontalListView. I have seen a few, but I know I wouldn't trust myself to make one so I am not sure how much I trust other peoples (the reason I would trust myself comes down to memory consumption and smoothness of the scroll).

    The Gallery, in my opinion is a good option, unless you find out why it was deprecated and it ends up being a show-stopper. It has some nice features no other AdapterView has (horizontal scroll and center-locking among them). I hate using deprecated API and hate even more suggesting someone else use one, but I haven't found a clean replacement for it...

    The ViewPager would be my second choice. It only lets you show one 'card' (which holds your view) at a time. There are ways around it, probably, by filling your card with multiple representations (i.e. show 5 columns of data per page/card). I will have to think more on that...
    Steve Luke
    Bartender

    Joined: Jan 28, 2003
    Posts: 4165
        
      21

    So I found a good horizontal scrollable view. The ViewPage with some work to it to make the elements to the side viewablea and passing some touch events around. Here is the implementation:
    https://gist.github.com/devunwired/8cbe094bb7a783e37ad1

    It was written by Dave Smith, author of Android Recipes (http://www.apress.com/9781430263227) a book I am quite fond of. I was able to modify it to made it look kinda close to what you might like:



    [Strum.jpg]

    Mike Riesen
    Greenhorn

    Joined: Apr 02, 2014
    Posts: 7
    Very neat, I'll have to look into that modified ViewPager. I had planned on spending the weekend coding, but the wife had other plans.
    I decided to set myself a milestone of getting the multiple saves working properly then start on a rewrite. I'm a very slow coder

    Thanks for all the advice, I might check back in if/when I hit a hit a snag.
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: Data storage and retrieval choices
     
    Similar Threads
    Design patterns
    JDBC Architecture
    Auxiliary tables saving
    how to read XML only once and provide data during whole session in data layer class
    Is HibernateCallback appropriate for executing SQL/procedures?