Popping in streaming audio

[Jeremy Tartaglia]

Not sure if this is the right place for this post, but it seems to be the best fit.

I created an audio core for a game, and it works fine, except on occasion there is a very faint pop. It's not present in the sound clips (which are in Ogg Vorbis format), and I'm using the VorbisSPI to decode them. The core works like this:

It creates an audio line for each sound to be played, and loads a buffer from the file. A thread is running in the background which then pipes as much data as the line can hold out of the buffer, and into the line, refilling the buffer from the file.

I'd post code, but I feel there's way too much to post, and it's spread out over a few classes. I thought the pops might have something to do with the VM being bogged down with creating and loading buffers, so I created a circular buffer for the decoded data, and it does pop less, but it still pops. I'm completely at a loss for ideas.

If you need or would like to see the code, I'll post a simplified version of it. I just don't have one right now.

Thanks in advance.

[Jim Yingst]

Yeah, I think it's hard to offer any useful suggestions here without seeing some code. Plus, refactoring your code for simplicity is usually a good thing anyway. Hopefully it will become easier for you to do further work on, as a result.

[Jeremy Tartaglia]

Well, I think my code is relatively well factored, other than perhaps to use the existing CircularBuffer class, but for my purposes, I think it would be overkill.

WARNING: This is unpolished, mostly uncommented code. Sorry.

/**
 * <p>Title: </p>
 * <p>Description: </p>
 * <p>Copyright: Copyright (c) 2006</p>
 * <p>Company: </p>
 * @author not attributable
 * @version 1.0
 */
 
import java.io.*;
import javax.sound.sampled.*;
import javax.sound.midi.*;
 
public class Sound
{
  private AudioFormat format;
  private AudioInputStream stream;
  private SourceDataLine line;
  private String name;
  private boolean paused;
  private boolean playing;
  private boolean ready;
  private boolean finished;
  private int pos;
  private int loops;
  private byte[] buffer;
 
  public static final int BUFFER_SIZE       = 24*1024;
  public static final byte[] ZERO           = new byte[BUFFER_SIZE];
 
  public Sound()
  {
    format=new AudioFormat(44100,16,2,true,false);
    name=null;
    paused=false;
    playing=false;
    ready=false;
    loops=0;
    pos=0;
    buffer=new byte[BUFFER_SIZE];
    finished=false;
  }
 
  public Sound(String name)
  {
    this();
    load(name);
  }
 
  public void setLoops(int i)
  {
    // Tells how many loops to play
    // -1 means loop forever
    // 0 means terminate after playing
    loops=i;
  }
 
  public int getLoops()
  {
    return loops;
  }
 
  public synchronized void load(String name)
  {
    AudioInputStream source=null;
    dispose();
    this.name=name;
 
    try
    {
      source=AudioSystem.getAudioInputStream(new File(name));
      stream=AudioSystem.getAudioInputStream(format,source);
    }
    catch (UnsupportedAudioFileException e)
    {
      e.printStackTrace();
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }
    catch (IllegalArgumentException e)
    {
      e.printStackTrace();
    }
    finished=false;
  }
 
  public synchronized void reload()
  {
    AudioInputStream source=null;
 
    try
    {
      source=AudioSystem.getAudioInputStream(new File(name));
      stream=AudioSystem.getAudioInputStream(format,source);
    }
    catch (UnsupportedAudioFileException e)
    {
      e.printStackTrace();
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }
    catch (IllegalArgumentException e)
    {
      e.printStackTrace();
    }
    finished=false;
  }
 
  public synchronized void loadData()
  {
    pos=0;
    if (name==null) return;
    if (finished==true) return;
    finished=!floodBuffer(buffer);
  }
 
  public synchronized boolean floodBuffer(byte[] buffer)
  {
    int offset=0;
    int numBytesRead=0;
 
    try
    {
      while (offset<BUFFER_SIZE)
      {
        numBytesRead=stream.read(buffer,offset,BUFFER_SIZE-offset);
        if (numBytesRead<0)  // EOF hit
        {
          if (loops==0)
          {
            System.arraycopy(ZERO,offset,buffer,offset,BUFFER_SIZE-offset);
            return false;
          }
          else
          {
            reload();
            if (loops>0) loops--;
          }
        }
        else offset+=numBytesRead;
      }
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }
    return true;
  }
 
  public synchronized void dispose()
  {
    stream=null;
    line=null;
    name=null;
    paused=false;
    playing=false;
    loops=0;
    pos=0;
  }
 
  public void pause()
  {
    paused=true;
  }
 
  public void resume()
  {
    paused=false;
  }
 
  public boolean isPlaying()
  {
    return playing;
  }
 
  public boolean isPaused()
  {
    line.stop();
    return paused;
  }
 
  public boolean isReady()
  {
    return ready;
  }
 
  public void play()
  {
    if (paused) return;
    playing=true;
 
    try
    {
      DataLine.Info info=new DataLine.Info(SourceDataLine.class,format);
      line=(SourceDataLine)AudioSystem.getLine(info);
      line.open(format);
    }
    catch (LineUnavailableException e)
    {
      e.printStackTrace();
      return;
    }
 
    loadData();
    line.start();
    ready=true;
  }
 
  public void stop()
  {
    ready=false;
    playing=false;
    line.drain();
    line.close();
  }
 
  public void outputSound()
  {
    int stored;
    int temp;
    if (!playing || paused) return;
    if (finished) playing=false;
    else
    {
      stored=line.available();
      if (stored+pos>=BUFFER_SIZE)
      {
        line.write(buffer,pos,BUFFER_SIZE-pos);
        temp=stored+pos-BUFFER_SIZE;
        while (temp>BUFFER_SIZE)
        {
          temp-=BUFFER_SIZE;
          loadData();
          if (finished) return;
          line.write(buffer,0,BUFFER_SIZE);
        }
        loadData();
        if (finished) return;
        line.write(buffer,0,temp);
        pos=temp;
      }
      else
      {
        line.write(buffer,pos,stored);
        pos+=stored;
      }
      if (pos>=BUFFER_SIZE) loadData();
    }
  }
 
  public static void main(String[] arg)
  {
    if (arg.length!=2)
    {
      System.out.println("Unknown file names!");
    }
    else
    {
      Sound s=new Sound(arg[0]);
      Sound p=new Sound(arg[1]);
      s.play();
      p.play();
      while (s.isPlaying() && p.isPlaying())
      {
        s.outputSound();
        p.outputSound();
 
        try
        {
          Thread.sleep(10);
        }
        catch (InterruptedException e)
        {
        }
      }
      System.exit(0);
    }
  }
}

[Jim Yingst]

I've only taken a cursory look at the code so far, and I'm not familiar with the sound APIs you're using. Yet. But - are the sounds your're playing all small enough to fit within the BUFFER_SIZE you're using? Or are we talking about ongoing streaming through these buffers? I'm guessing the latter (from the thread title) but want to confirm. Do the clicks occur regularly? Is it possible they're occurring specifically whenever you refill the buffer? Or as least, sometimes when you fill the buffer? I.e. do clicks occur more often than buffer refils, less often, or exactly the same? Adding a println("fililng buffer") could give you a good idea which is the case - just look at the output as you listen to the sound.

Looking at floodBuffer(), my spidey sense is going off when I see the code is filling the buffer entirely. I'm thinking it may well be smoother to just read as many bytes as the system has ready, and pass them on to the AudioSystem.getLine() as they arrive, rather than making sure you fill up the buffer first. If theres some reason the system can't fill the buffer in one shot, you might as well deliver the bytes you do have, and then come back for more. That's what I'd be inclined to try, anyway. Though again, I'm not familiar with AudioStream etc, so maybe there's some reason why this approach doesn't work.
[ February 05, 2006: Message edited by: Jim Yingst ]

[Jeremy Tartaglia]

But - are the sounds your're playing all small enough to fit within the BUFFER_SIZE you're using? Or are we talking about ongoing streaming through these buffers?

Yeah, these files are very large, even compressed: 2-4 megabytes ~ 20-50 uncompressed.

Do the clicks occur regularly? Is it possible they're occurring specifically whenever you refill the buffer? Or as least, sometimes when you fill the buffer? I.e. do clicks occur more often than buffer refils, less often, or exactly the same? Adding a println("fililng buffer") could give you a good idea which is the case - just look at the output as you listen to the sound.

The pops occur at the same instances every time, but it's not after a regular interval, no. I tried the System.out.println, and floodBuffer has nothing to do with the pops. They occur much more frequently than the pops.

Looking at floodBuffer(), my spidey sense is going off when I see the code is filling the buffer entirely. I'm thinking it may well be smoother to just reas as many bytes as the system has ready, and pass them on to the AudioSystem.getLine() as they arrive, rather than making sure you fill up the buffer first.

This was how I had it the first time. But, it requires that you have to read bytes from the file before you output them to the line. This creates IO lag, while my system loads the data from the file AFTER data has already been output to the line. It pops less now, but still pops.
[ February 05, 2006: Message edited by: Jeremy Tartaglia ]

[Jeremy Tartaglia]

You know, maybe I should just never post questions here again.

The code is fine. The converter I used to go from ACM WAV -> Ogg Vorbis screwed up. The pops are in the audio file.

Everyone point and laugh at the dumb rookie.

Thanks for trying to help though.

[Jim Yingst]

Interesting. I'm surprised that the problem diminished when you switched to filling the buffer, but OK. Is it possible there was just some bug in the original code, that went away when you rewrote it to fill the buffer?

In main(), it looks like you're trying to stream two different sounds in one loop. I wonder if sometimes one sound is taking up too much of the time, while the other needs more data? Perhaps you could create two different Threads, each one feeding bytes to a different Sound object. That way while one thread's busy trying to fill its buffer reading from the sound file (which may well involve a blocking read), the other thread has a chance to do something and maybe keep the second sound from running out of data to play.

You might also want to play around with the buffer size and even the length of the Thread.sleep(), if you haven't already.

We don't get a lot of questions about the sound API here. In theory Other Java APIs would be the best place to ask about that, though in practice I'm not sure there are a lot of posters familiar with Java sound in that forum either. Might be worth a try though.

[Jim Yingst]

D'oh! I saw your last message just after posting the above. Well, on the plus side I learned a little bit about the sound APIs, about which I knew nothing, previously. Now I know... slightly more than nothing. So no complaints here. Cheers...