Broadcasting video with Android - without writing to local files

Submitted by kisg on Wed, 07/08/2009 - 20:27

Android

One of the weaker points of the Android platform is the Media API. When compared to the J2ME API, one important feature is missing: the ability to record to a stream and to playback from a stream.

Why is this important? There are a number of use cases.

For recording:

post-processing audio / video data before writing out to the file system
broadcasting audio / video without writing out the data first into the file system, which also limits the broadcast to the available free space on the device.

For playback:

pre-processing the audio / video data before playing
streaming using protocols that are not supported by the built-in media player

In this blog entry we will show a method to broadcast video (and audio) from an Android phone to a network server, without writing to the file system.

There is one promising method in the MediaRecorder class setOutputFile(FileDescriptor).

We know that in Linux also network sockets have file descriptors. But how could we access the file descriptor of a regular java.net.Socket?

Luckily, ParcelFileDescriptor comes to the rescue, where we can use the fromSocket(Socket) static method to create a ParcelFileDescriptor instance from a Socket object. From this instance, we may now grab the badly needed FileDescriptor.

It all boils down to these few lines (in pseudocode):

String hostname = "your.host.name";
int port = 1234;

Socket socket = new Socket(InetAddress.getByName(hostname), port);

ParcelFileDescriptor pfd = ParcelFileDescriptor.fromSocket(socket);

MediaRecorder recorder = new MediaRecorder();

// Additional MediaRecorder setup (output format ... etc.) omitted

recorder.setOutputFile(pfd.getFileDescriptor());

recorder.prepare();

recorder.start();

Using this concept we created a small proof of concept application (together with an even simpler server), which is able to broadcast videos not limited in length by the available space on the SD Card.

There are a few gotchas, if you want to try this out yourself:

The MediaRecorder records either in 3GPP or in MP4 format. This file format consists of atoms, where each atom starts with its size. There are different kinds of atoms in a file, mdat atoms store the actual raw frames of the encoded video and audio. In the Cupcake version Android starts writing out an mdat atom with the encoded frames, but it has to leave the size of the atom empty for obvious reasons. When writing to a seekable file descriptor, it can simply fill in the blanks after the recording, but of course socket file descriptors are not seekable. So the received stream will have to be fixed up after the recording is finished, or the raw video / audio frames have to be processed by the server.
For some reason, the MediaRecorder also leaves the header of the file blank, which also has to be handled on the server.
High latency connections will cause the video to be choppy. Obviously some buffering is necessary. One method is to use a local mini server on the phone which receives the stream, buffers it, and sends to the remote server as fast as the network allows it. However, if using native code is an option, we can simply create a pipe to receive the data from the MediaRecorder. We will show this method in a future blog entry.

Trackback URL for this post:

http://www.mattakis.com/trackback/30

kisg's blog

Comments

very usefull

Submitted by zeeshan (not verified) on Thu, 07/09/2009 - 10:23.

very usefull material,
thanks

I could use this almost

Submitted by ruzzel01 (not verified) on Wed, 09/01/2010 - 13:31.

I could use this almost everyday.
tampa seo

Hi kisg, can you provide its

Submitted by zeeshan (not verified) on Fri, 07/10/2009 - 12:17.

Hi kisg,

can you provide its full working source please?

really appreciate your help

Thanks for sharing great

Submitted by Anonymous (not verified) on Mon, 07/13/2009 - 14:27.

Thanks for sharing great technique..

obviously very few people will know abt socket to fd.

thx
suds

Any tips for server side?

Submitted by garancsi (not verified) on Mon, 07/27/2009 - 13:31.

Can You please give some tips for the server side?
What are you using to capture/broadcast the stream?

thx
T

hi i have try to receive and

Submitted by hi (not verified) on Fri, 07/31/2009 - 05:11.

hi i have try to receive and play by MediaPlayer using this way

but i can't setup and play it.

could you give me some hint for that?

Socket socket = new Socket(serverAddr, PORT); //use tcp to receive
pfd = ParcelFileDescriptor.fromSocket(socket); //set pfd
MP = new MediaPlayer();
MP.setAudioStreamType(AudioManager.STREAM_MUSIC); //create MediaPlayer
MP.setDataSource(pfd.getFileDescriptor()); //set datasource by pfd
MP.prepare();
MP.start();

do i miss to set something?

thx

the same

Submitted by Anonymous (not verified) on Tue, 10/13/2009 - 15:33.

i habe the save problem.

do u have got it out now?

thanks

Friend have you find any

Submitted by Parag Patel (not verified) on Sat, 01/30/2010 - 18:53.

Friend have you find any solution ?? plz let me know if you have.

Have u found its working.

Submitted by Parag Patel (not verified) on Mon, 02/01/2010 - 04:58.

Have u found its working, If yes then plz share it. I am get stuck at same point.
Thanks,
Parag Patel

Good post, but I have one more question...

Submitted by Maurizio (not verified) on Tue, 08/04/2009 - 15:35.

Hi all,

really useful post... I would like to add one more element. I also need to stream raw data without any form of video encoding... is it possible to do???

Thanks and how about DatagramSocket?

Submitted by Mark Spencer (not verified) on Fri, 08/07/2009 - 06:24.

Thanks for this wonderful post. I've been pulling my hair out on this for hours now. Is it possible to do a similar trick with a datagram socket?

Mark

Trying to get this to work

Submitted by Pat (not verified) on Mon, 08/24/2009 - 17:26.

Hi kisg,

I've been trying to get this to work for a few weeks now... Do you have more information/documentation or code I could look at? It would be incredibly helpful if you could post them on the site or send them to me. Thanks.

Pat

Great

Submitted by Stefano (not verified) on Sat, 09/26/2009 - 20:02.

Great kisg,

please more details and snipped code, could be a success for all.
Thanks
Stefano

"For some reason, the

Submitted by Anonymous (not verified) on Wed, 11/11/2009 - 13:24.

"For some reason, the MediaRecorder also leaves the header of the file blank, which also has to be handled on the server."
Any ideas how to handle this?
Is there a posibility to generate the header on the Server?

Awesome! Works flawlessly and

Submitted by Filipe Freitas (not verified) on Thu, 12/10/2009 - 15:48.

Awesome!
Works flawlessly and smooth!
I didn't even had to fix the stream, as the file generated on the server is working.
My previous attempt at this was to create 10s files and send them.

Repairing the Output

Submitted by thedracle (not verified) on Tue, 01/26/2010 - 18:07.

Just to help those having issues, the SDK seems to try to seek to insert the size values of the mdat atom, and also the moov header.

I set the encoder in this example to produce a THREE_GPP file.

In order to play the output THREE_GPP, you're going to need to first create the header in the first 28 bytes prior to the mdat atom (which should all be zeros).

00 00 00 18 66 74 79 70 33 67 70 34 00 00 03 00 33 67 70 34 33 67 70 36 00 02 F1 4D 6D

The 6D is the 'm' first byte in the mdat atom. The four bytes proceeding that need to be modified to include the integer value of the byte in your stream containing the output moov atom (Which should be output upon stopping the recording). As long as this header is correctly set, and the player can locate the moov atom- everything should play back correctly.

Also, the socket method here isn't very flexible- you can perform finer alterations of the packet data to a network (I'm attempting this at the moment for live streaming), by providing it with a local socket, and then connecting to that local socket and processing its output independently (In a thread for instance) for transmission over UDP, RTP, etc..

- Jason

Play incoming stream

Submitted by Parag Patel (not verified) on Mon, 02/01/2010 - 05:10.

Can we do same thing (user local socket) to receive incoming RTP stream which has H326 encodec data?

I have successfully send RTP stream, but when same way going to receive incoming RTP stream, MediaPlayer setDataSource function goes fail. giving me "offset error". I have looked Android MediaPlyer code, its giving error at below point.

status_t MediaPlayerService::Client::setDataSource(int fd, int64_t offset, int64_t length) { LOGV("setDataSource fd=%d, offset=%lld, length=%lld", fd, offset, length); struct stat sb; int ret = fstat(fd, &sb); if (ret != 0) { LOGE("fstat(%d) failed: %d, %s", fd, ret, strerror(errno)); return UNKNOWN_ERROR; } LOGV("st_dev = %llu", sb.st_dev); LOGV("st_mode = %u", sb.st_mode); LOGV("st_uid = %lu", sb.st_uid); LOGV("st_gid = %lu", sb.st_gid); LOGV("st_size = %llu", sb.st_size); if (offset >= sb.st_size) { LOGE("offset error"); ::close(fd); return UNKNOWN_ERROR; }

Can any one help me to understand what I am doing wrong?

Thanks,
Parag Patel

It needs to understand the format.

Submitted by thedracle (not verified) on Thu, 02/04/2010 - 22:02.

Parag,

I'm trying to the same thing now- setting the data source to an fd with RTP data on it isn't going to work- since it has no ability to automatically detect the format type. There's no way to set this manually either :( MediaPlayer does support RTSP though, and it should be possible to send it the right SDP information to accomplish this.. I'm amazed nobody else has done this as far as I can tell...

- Jason Thomas.

Source

Submitted by Anonymous (not verified) on Thu, 04/15/2010 - 10:44.

Hi Parag,

Can you please share the code for sending RTP stream?

Thanks,
Anu

"Just to help those having

Submitted by Anonymous (not verified) on Mon, 08/02/2010 - 17:11.

"Just to help those having issues, the SDK seems to try to seek to insert the size values of the mdat atom, and also the moov header.

I set the encoder in this example to produce a THREE_GPP file.

In order to play the output THREE_GPP, you're going to need to first create the header in the first 28 bytes prior to the mdat atom (which should all be zeros).

00 00 00 18 66 74 79 70 33 67 70 34 00 00 03 00 33 67 70 34 33 67 70 36 00 02 F1 4D 6D

- Jason"

Thank google (and you, Jason) for this!

I merely wanted to stream the MediaRecorder data immediately to a server for storing a usable file, but was obviously unsuccessful due to missing header information. Upon fixing up the file as you indicated, I got what I wanted.

While it can't be played as a stream due to the moov box data not preceding the mdat box (apparently a requirement for streaming player reception), that is not my current need.

sample code in sipdroid.org

Submitted by yu.xiao (not verified) on Sat, 02/06/2010 - 17:22.

you can get the sample code in VideoCamera.java in Sipdroid.org.

as for playback, it seems that android mediaplayer can not play rtp stream for lacking sdp information. if I am wrong, please correct me.
thanks.

Play the stream

Submitted by Roger (not verified) on Thu, 04/08/2010 - 20:11.

Hi folks,

I think I made some code that works, with some help from this page and looking at the code of the sipdroid. Now I want to play this stream in my computer, but I don't know how. It tried with VLC but I couldn't make it work. I am using ubuntu 9.04. Someone have some tips for me? If you are interested in the code I can paste it here.

Regards,

Roger

Hi Roger, Sorry, don't have

Submitted by Anonymous (not verified) on Wed, 04/21/2010 - 23:53.

Hi Roger,

Sorry, don't have any tips for you, but would love to see working code for this process.

Thanks!
Scott

nice work

Submitted by Zag (not verified) on Fri, 05/14/2010 - 13:06.

you say J2ME can stream video to a server but I dont think it can?

The android code would be most useful but I will try this later, thanks for the great post.

Hi, You can specify

Submitted by kisg on Wed, 05/26/2010 - 11:24.

Hi,

You can specify setOutputStream for the RecordControl. Then you can take this stream and send it to a remote server through the network.

Best Regards,
Gergely

There's no

Submitted by Winters Li (not verified) on Tue, 06/08/2010 - 06:47.

There's no "setOutputStream()" method for "MediaRecorder"

Hi, coudl you guys (blogger

Submitted by Anonymous (not verified) on Wed, 05/26/2010 - 18:32.

Hi, coudl you guys (blogger or you others) that made it work all the way) so you can see whats being captured on the pc, share the code please?

Im having troubles figuring how to handle the stream at the other end (server)

I got the same troubles in

Submitted by Winters Li (not verified) on Sat, 06/05/2010 - 04:09.

I got the same troubles in the other end too.

Android phones are the best.

Submitted by Lilia (not verified) on Mon, 06/21/2010 - 14:27.

Android phones are the best. When will iPhone renounce its software and use Android instead?
Cheap VPS

009 来祝贺

Submitted by wow gold (not verified) on Mon, 06/28/2010 - 07:16.

We are sell wow gold and wow power leveling and wow gold or knight noah

rolex

Submitted by Anonymous (not verified) on Thu, 07/08/2010 - 07:06.

Carol Mueller said replica watches her husband remained replica consistently devoted tag heuer to this project fake rolex After coming up rolex with the idea

watches

Submitted by Anonymous (not verified) on Thu, 07/08/2010 - 07:06.

spent every weekend replica watches working on it fake watches He would be out there watches all day Saturday replica rolex and all day Sunday cartier she said.While Keith breitling did most of the work

cheap NFL Jerseys Bastian

Submitted by Anonymous (not verified) on Wed, 07/14/2010 - 07:18.

cheap NFL Jerseys
Bastian Schweinsteiger Jersey
Schweinsteiger Jersey
Lukas Podolski Jersey
Podolski Jersey
Miroslav Klose Jersey
Klose Jersey
Thomas Mueller Jersey
Mueller Jersey
Michael Ballack Jersey
Robin Van Persie Jersey
V.Persie Jersey
Arjen Robben Jersey
Robben Jersey
Wesley Sneijder Jersey
Sneijder Jersey
Andres Iniesta Jersey
A.Iniesta Jersey
David Villa Jersey
Xavi Hernandez Creus Jersey
Xavi Jersey
Fernando Torres Jersey
Torres Jersey
Cesc Fabregas Jersey
Fabregas Jersey
Sergio Ramos Jersey
Atlanta Braves jerseys
Baltimore Orioles jerseys
Boston Red Sox jerseys
Chicago Cubs jerseys
Chicago White Sox jerseys
Cincinnati Reds jerseys
Cleveland Indians jerseys
Detroit Tigers jerseys
Houston Astros jerseys
Houston Colts Jerseys
Kansas City Royals jerseys
Los Angeles Angels jerseys
Los Angeles Dodgers jerseys
Milwaukee Brewers jerseys
Minnesota Twins jerseys
Montreal Expos jerseys
New York Mets jerseys
New York Yankees jerseys
Oakland Athletics jerseys
Philadelphia Phillies jerseys
Pittsburgh Pirates jerseys
San Diego Padres jerseys
San Francisco Giants jerseys
Seattle Mariners jerseys
Seattle Pilots Jerseys
St.Louis Cardinals jerseys
Tampa Bay Rays jerseys
Texas Rangers jerseys
Toronto Blue Jays jerseys
Washington Nationals jerseys

AARON CURRY jersey Aaron

Submitted by lijunfuck (not verified) on Sat, 08/28/2010 - 04:45.

AARON CURRY jersey
Aaron Kampman jersey
Alan Page jersey
ALBERT HAYNESWORTH jersey
Antonio Cromartie jets jersey
Bobby Hebert Jersey
Brian Westbrook 49ers Jersey
Bruce Smith jersey
C.J. Spiller Jersey
Chester Taylor bears Jersey
CORTLAND FINNEGAN jersey
Devery Henderson Jersey
Dustin Keller jersey
Eric Berry Jersey
Gerald McCoy jersey
HARRY CARSON jersey
Hines Ward jersey
HOWIE LONG jersey
Ike Taylor jersey
Jahvid Best jersey
JAKE LONG jersey
James Farrior jersey
Jared Odrick Jersey
Jay Ratliff Jersey
Jerome Bettis jersey
Joe Flacco jersey
JOE KLECKO jersey
Joe Theismann jersey
Kareem Jackson Jersey
Karlos Dansby dolphins jersey
KENNETH MOORE Jersey
Kory Sperry Jersey
KYLE ORTON jersey
LAVERANUES COLES jersey
Lofa Tatupu jersey
MARK BAVARO jersey
MARK DUPER Jersey
MARK GASTINEAU jersey
Marques Colston Jersey
Mike Bell Jersey
Mike Iupati jersey
Morten Andersen Jersey
PHIL SIMMS jersey
Roger Staubach jersey
Santonio Holmes Jets Jersey
SONNY JURGENSEN jersey
Stefan Logan jersey
Steve Atwater jersey
STEVEN JACKSON jersey
Terence Newman jersey
TIM HIGHTOWER jersey
Tony Dorsett jersey
TRACY PORTER Jersey
Tyson Alualu Jersey
VINCE YOUNG jersey
BOB SANDERS jersey
Brandon Marshall Dolphins Jersey
Broncos Tim Tebow Jersey
Chad Henne jersey
Charles Woodson jersey
Dan Williams Jersey
DARREN McFADDEN jersey
Darren Sharper Jersey
Demaryius Thomas Jersey
Dez Bryant Jersey

" All shows are archived

Submitted by replica watches (not verified) on Mon, 07/19/2010 - 10:28.

" All shows are archived tissot on Hulu for viewing.
Click Here replica watches for the Blotter Homepage.
Gun breitling aeromarine Sales Expected to Shoot Up
Supreme tissot quadrato chronograph Court Ruling Seen as Windfall tudor women for Firearms Manufacturers, Dealers
By RICH knock off jewelry BLAKE
July 16, 2010—
After a prs200 Supreme Court decision affirming the piaget replica right to bear arms was dior jewelery handed down in June, the gucci bracelets owners of Midwest Sporting Goods, rado watches for men just outside of Chicago, started baume & mercier riviera noticed something any retailer

Nice to hear about

Submitted by ruzzel01 (not verified) on Thu, 08/12/2010 - 09:33.

Nice to hear about it.
vehicle tracking

Wholesale NFL jerseys and

Submitted by wholesale nfl jerseys (not verified) on Wed, 08/18/2010 - 03:46.

Wholesale NFL jerseys and retail wholesale nfl jerseys
,NHL jerseys,MLB jerseys,NBA jerseys,hockey jerseys,Throwback football jerseys,we also wholesale jerseys
wholesale cheap
NFL jerseys,wholesale mlb jerseys
-Welcome to the best wholesale NFL jerseys shop,we are one of the largest NFL jerseys wholesaler on the web.

I would like to say thank you

Submitted by jonli (not verified) on Sun, 08/22/2010 - 22:08.

I would like to say thank you to author of these articles on this site. I read all of these articles and i need to read some new articles. I've watched a video on youtube about this topic for now and i loved it. Also it is one of the rarely topic on this site.

See you on a new topic...