Thats what the whole concept of Agile Estimation lies..
Obviously any two people will have different speeds of doing work.
Simple dynamics, So if you are going to estimate time, you would need speed and speed varies person to person.
Estimate in size and relative complexity or past experience and assign a weight or story point to it.
Have every one involved in this process, Use planning poker if you feel.
Then break up the user stories into tasks.Perhaps each person can take up tasks and estimate his/her time.
First couple of sprints you can give the levy to the team to find the velocity range then you can plan your sprints further.
Watch this video by Mike Cohn at google :
http://www.youtube.com/watch?v=fb9Rzyi8b90