G'day folks, for fear of being flamed for asking an open ended question on StackOverflow, I thought I'd bring it here instead and hope to muster up a good discussion.
I'm getting to the stage of my android learning path where I want to start testing the applications that I create, and so far this is what I believe I'll need
Unit testing - Testing the smallest parts of my code, preferably down to single methods. For example if my application is a calculator, and I have a method to add(int val1, int val2) then I would want a unit test(s) around that method.
Integration testing - Similar to standard unit testing, however these should test "features" of the application rather than units of code. For example I'd want to test the addition functionality of my application by inputting some values, clicking a button, and verifying the response.
What I'm thinking about, is what should I be testing? Here is what I've come up with so far, can anyone please advise me if I'm on the right track, or what I should add?
Test business logic, such as single methods in each of my classes (not overridden android methods, but my own ones)
Test that when I click one button it does actually take me to another activity, for example if my "main" page has a menu, and I click a button to "start game", it actually takes me to the start game screen.
Test that I can load a set of data from a SQLite instance (not sure if this is better placed in UnitTesting, however it would require access to android classes)
My application is a 5 question multiple choice quiz, test from start to finish that this can be completed along with validation tests (making sure you can't move to question 2 if you haven't answered question 1)
Test that buttons actually appear on the screen (visibility checks)
Test that TextViews contain the same text as their resource counterparts in strings.xml
I've been Googling for a few days now, I can find various tutorials / blog posts about peoples experiences working with Robotium/Roboelectric etc, however I can't find anything that specifically advises what should or shouldn't be tested.
I always layer my apps so that I separate those parts that interact with the framework and the parts that implement business logic or algorithms. The latter are easy to unit test since I don't have to worry about providing the framework (or a mock thereof).
Thanks for the comments (and link, bookmarked for a rainy day ;) )
Can you expand a little more on what you mean by layering? I'd like to understand a little more about how you do that, as I think it might be able to help me.
Are you suggesting something along the lines of keeping my activities as streamlined and cut down as possible, and only keeping "android" specific code there, and doing all my business logic outside of any class that extends from Activity? That is something I've thought of, however theres not much I can move out since pretty much everything I do will involve interacting with the android framework, such as reading/writing to SQLite, manipulating UI etc.
Also, how detailed do you tend to take your testing, do you test that buttons are on the screen, have the right labels, check that textviews contain the right text etc? Or is that slightly too pedantic?
a) I refactor my code such that any code that interacts with the framework (the Android SDK in this case) is in one class and all of my algorithms are in another class. I can the easily unit test the class that doesn't need the framework.
b) Similar to (a), but I move the refactored algorithms to separate methods within the same class. I usually do this is I have only an algorithm or two that I need to test. And this works well for unit testing Android code, even Activities, because the Android classes provide in the SDK are just stubs and don't do anything, thus there is no need to set up anything (like you might have to do with other frameworks).
c) I do something like a mock object: I pull out calls to framework methods into separate methods in my code. Then I extend the class in my unit test and mock the framework methods.
subject: Android testing, what *should* I be testing?