• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

running application in C on mobile phone with Android

 
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello :-)!

SUMMARY: How to run pocketsphinx_continuous on mobile phone with Android, and how to do it with simulator of mobile phone with Android. PocketSphinx can be downloaded for free here, for both Linux and Windows: http://cmusphinx.sourceforge.net/html/download.php .



------------------------------------------------------------------------------------------------------------

SHORT EXPLANATION

I'd like to create application for mobile phone and server such that:
1. User runs application on mobile phone with Android, this application uses PocketSphinx. It is Automatic Speech Recognition software for devices like mobile phones, written in C. Somebody told me that "Pocketsphinx is used successfully on Symbian with minimal effort". (However I think Android may be better choice). My knowledge about Android is limited but I know something about CMU Sphinx. Can you help me, please, to run this PocketSphinx on Android? I also need to find simulator of mobile phone with Android. I have already found Wireless Toolkit and I tried to use this emulator, however I think it doesn't allow to run simulation of mobile phones with Android. (By the way I am limited in my project to mobile phone of the price up to about 160 euro; which Android version would be best for my application?).
2. This application communicates with the user. The users speaks digits and some other words (dictionary is about fifteen words, I can create language and acoustic models, as well as formal grammar in CMU Sphinx, with the use of SphinxTrain). The application recognizes those digits.
3. Based on this talk, mobile phone sends information about results of recognition to the server. I can do it in Java for CLDC/MIDP with httpconnection, POST method on the mobile phone side and with Tomcat on server. I think this httpconnection must be possible not only in Java ME, but also in Android C. But this is thing which will bother me later, now I'd like to run pocketsphinx_continuous on Android and try to modify it according to my needs.

Summing up, what I'd like to know is how to run demo pocketsphinx_continuous on mobile phone with Android, and how to do it with simulator of mobile phone with Android. (I've got Wireless Toolkit but I think it is not capable of running Android). PocketSphinx can be downloaded for free here, for both Linux and Windows: http://cmusphinx.sourceforge.net/html/download.php . There are two ways of installing it in Ubuntu. First is to unpack it (it is tar.gz), enter its directory and type "./configure", "make", "sudo make install". The other, different, is written in README file, i.e. run autogen.sh, "./configure", "make clean all", "make test", "make install". The result should be, among many other files, javadoc in doc directory.

------------------------------------------------------------------------------------------------------------

LONG EXPLANATION

What is required to give me some advices is written above. So if you don't want / don't have time to read the whole text, just above explanation is enough. But let me say about other possible approaches which I try to do. Those are worse than what I explained above, but I also tried them.

CMU Sphinx contains SphinxTrain to create acoustic model. It has got some different recognizers (also called decoders), those are PocketSphinx, Sphinx4 and some other. PocketSphinx is written in C and is for devices like mobile phones. Sphinx4 is Java application for PCs because it requires better hardware. There is good tutorial about using SphinxTrain with Sphinx4. It is here: http://www.speech.cs.cmu.edu/sphinx/tutorial.html . There is no similar tutorial for PocketSphinx. However with some knowledge about Android and C it is possible to run it on Android. The other approach than PocketSphinx, is using this Sphinx4. However it requires connection between mobile phone and server, sending audio data from cellular phone to server. It can be done with Skype, but there is still problem on server side with redirecting sound from Skype to Sphinx4. So you see there are two different approaches. One is to run speech recognition on mobile phone with PocketSphinx and Android. The other is speech recognition on server with Sphinx4. There are some ways to follow this second option. One is Skype on both server and mobile phone. (But there is problem with redirecting speech from Skype to Sphinx4). The other is too expensive Digium card. The third one is IVR, but I still look for good open-source IVR software. All of those three ways of second option involve Sphinx4. It was much easier to create application with Sphinx4. It is for PCs, not embedded devices, because it requires better device, extensive floating point math and some other things. But it has much better documentation than PocketSphinx. But I want to use PocketSphinx because I don't need to pay to anybody for access to mobile internet in order to send data through internet for application like Skype. The access to internet is required only for short time to send little text data with httpconnection, POST method. This is why I'd like you to help me, please, with running pocketsphinx_continuous on simulator of PocketSphinx on mobile phone with Android. (By the way, I see I've got installed S60 Developer Tools -> 3rd Edition FP1 SDK -> MIDP -> Emulator).

Summing up in a list, those are in CMU Sphinx:
I. SphinxTrain, which I can use to create files needed for speech recognition
II. decoders which use speech input and some files which I create with SphinxTrain, to follow speech recognition (input speech + files from SphinxTrain = are used by decoder)
II-1. PocketSphinx, written in C, for devices like mobile phones
II-2. Sphinx3, written in C, for PCs, actually the best developed
II-3. Sphinx4, written in Java, for PCs, with the best documentation
II-4. Some other, older versions

Those are possible approaches of solving my task:
I. Speech recognition on mobile phone with PocketSphinx. This is much better way than [II].
Problems: 1. running PocketSphinx on mobile phone with Android. 2. simulating mobile phone with Android on PC. I also thought that maybe I should consider Meamo, what do you think about it? Or use things different than CMU Sphinx, e.g. Simon, HTK, Julius. I looked at those other ASR engines and I think PocketSphinx may be best choice.
II. Speech recognition on server with Sphinx4. There are some different ways of establishing voice connection between mobile phone and server.
II-1. ordinary call from mobile phone to server; server has got Digium card. Disadvantage: Digium card is expensive.
II-2. to use Skype on both mobile phone and server. Problem: redirecting of sound from Skype to Sphinx4 and vice versa.
II-3. can you think about any other ways? I found that Asterisk may be useful.

Greetings :-)!
 
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Android, by definition, is Java. If you code in C, you'd be talking to the copy of Linux that's underneath Android. Of course, since C isn't "write once, run anywhere", that means that your code may be very hardware specific, since although I think all current Android devices are ARM-based, there's no guarantee, and even the underlying Linux OS's are subject to variation.

I haven't investigated what, if any JNI support the Dalvik JVM provides for native-code classes. Since TTS is primarily mathematical, it might be worthwhile porting the code from C to Java, as the two languages are so similar in that area. Although if there's any chance of piggybacking on a DSP, it might provide some performance boost.

On my own system, they cheat. Speech recognition is done by piping the digitized audio up to Google's speech-recognition servers. They're pretty impressive, but I'll admit that I wouldn't mind a local option. For one thing, I used to be able to define my own arbitrary voice-dial and program launch commands, not be constrained to what Google though would be good.
 
Johny Wyenski
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for your answer :-)!

Hm, I've got dilemma. Somebody told me that Symbian is widely used for commercial software so people who develop for Symbian don't want to share their knowledge too easily. On the other hand man from CMU Sphinx forum told that "Pocketsphinx is used successfully on Symbian with minimal effort" (http://sourceforge.net/projects/cmusphinx/forums). Now you say that "Android, by definition, is Java". This application which I'd like to run (PocketSphinx) is in C, not in Java. On the other hand, that man has written that it can be used with minimal effort. For me it looks like porting for Symbian or Android is relatively difficult thing. Are you sure there is no simpler way of running it than porting?

This PocketSphinx is generally for embedded devices, so not for any exact model of phone. There are two constraints, first is price (max 160 euro), second is hardware (http://cmusphinx.sourceforge.net/sphinx4/doc/Sphinx4-faq.html#j2me). I still need to choose which mobile phone would be best choice.

You say that TTS is primarily mathematical. PocketSphinx is Speech to Text (Automatic Speech Recognition), not Text to Speech (synthesizer). But it also involves some sophisticated Maths.

I'm not familiar with terms JNI, Dalvik JVM (however what is JVM and KVM of course I know) and piggybacking on DSP. I see that Java Native Interface allows to run Java code in JVM with use of C or other libraries. I don't see any need for this but maybe I miss something. PocketSphinx is C so I don't see any need for Java. And if porting to Symbian/Android also requires changing from C to Java before porting, isn't it too much to be done? Other thing, what is this piggy-backing on a DSP? There are too many references for piggybacking on Wiki .

May I ask you about simpler thing, not strictly connected with Android? I've got my application in Sphinx4. There are also examplary appications for Sphinx4 and PocketSphinx. I can run demos from PocketSphinx (as explained in first post), as well as demos from Sphinx4. I can change source code of demos from Sphinx4 to perform my task. I'd like to do similar thing with PocketSphinx, but even if it should be easy task, I cannot do it. In Sphinx4 it is much simpler because all source files are in one directory. In PocketSphinx it is done in somehow different way. (http://cmusphinx.sourceforge.net/html/download.php). Without ability to move my application from Sphinx4 there is no need for me to port this PocketSphinx to Android/Symbian.

Maybe this cheating would also be good idea for me. However I need to use it for Polish language which probably is not available. How to digitize audio and where to find those Google's speech-recognition servers?

Greetings :-)!
 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
OK, I'll try and gather up some of that stuff and clarify. Dalvik is the JVM that runs Android applications. Symbian is the proprietary (formerly proprietary, now open-sourced) OS that runs a large number of the mobile phones on this planet, especially in Europe. Symbian is programmed in C, so as long as you're running generic C code with no OS or hardware dependencies, it would be more or less compile-and-go.

Theoretically, Android could be the OS itself on Android devices. But it was easier to take an OS that was already widely ported (Linux) and piggyback Android on top of that than to run off custom Dalvik implementations for each possible device. Presumably one could just as easily put the Dalvik JRE in Symbian, but as far as I know, no one's planning to.

Incidentally, Android is fairly paranoid, so there's one JVM instance per Activity. Activities therefore don't share memory.

A DSP is a Digital Signal Processor, and as its name indicates, it's hardware that is optimized to do high-performance analysis and transformation of audio, video and other signal data. So if you're doing speech recognition, it's a good thing to have.

Sorry about being sloppy with terminology on TTS/ASR.

JNI is the Java Native Interface, and is the "rat hole" that allows Java code to call functions that are outside the JVM. Typicallly in C or assembly language and normally used for performance (not as much these days) and when it's necessary to talk directly to the hardware.

I just did a little checking. There is a JNI interface for Dalvik, although some rumors that JNI may be disabled (remember, Android is fairly paranoid), so it might require a "jailbroken" phone. But there is the potential to be able to link in the code you want. Just no guarantee that it will be something that can be made to work easily on every Android phone on the street.

 
Tim Holloway
Saloon Keeper
Posts: 27752
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Missed one:

Maybe this cheating would also be good idea for me. However I need to use it for Polish language which probably is not available. How to digitize audio and where to find those Google's speech-recognition servers?



This is an interface/app that Google themselves provides. I think its actually Google Voice. They know where their servers are, of course! I don't know what languages they currently support, however.
 
Rancher
Posts: 1369
1
Android Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have not worked on C but I thought of bringing this to your notice nevertheless and hope that this info is of some use to you. Besides, I am reading up on C and perhaps will "do" something sooner. I think, the discussion here will definitely help in one way or other.

In June this year, Google announced a NDK(Native development kit) that gives developers ability to call into native code from your Android applications.(See the announcement here or download the kit from here). They also have a google-group which you might want to revert to.

As regards JNI support:

Currently we support JNI shared libraries written in C or C++ that link with the Android versions of libc and libm and OpenGL ES.
In a future release we hope to support linking with audio libraries.

 
Johny Wyenski
Greenhorn
Posts: 21
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you for your answers :-)!

It looks like I still need to learn much about programming for mobile phones. Can you suggest me any book so that I can understand these issues better (preferably betterworldbooks.com or similar site).

Just yesterday somebody has written on CMU Sphinx forum that "Symbian part was yesterday committed to pocketsphinx, you see it's a few required modules for Nokia SDK: http://cmusphinx.svn.sourceforge.net/viewvc/cmusphinx?view=rev&revision=9470 ". So it looks like they work exactly at this moment on porting PocketSphinx to Symbian. Can those things which are to be downloaded on that page also be used for Android? Or do I need eventually to choose rather Symbian than Android?

I still work on what I have written fifth paragraph of my second post (beginning with "May I ask you about simpler thing").

Greetings!
 
reply
    Bookmark Topic Watch Topic
  • New Topic