Broadcasting video with Android - without writing to local files

in

One of the weaker points of the Android platform is the Media API. When compared to the J2ME API, one important feature is missing: the ability to record to a stream and to playback from a stream.

Why is this important? There are a number of use cases.

For recording:

  • post-processing audio / video data before writing out to the file system
  • broadcasting audio / video without writing out the data first into the file system, which also limits the broadcast to the available free space on the device.

For playback:

  • pre-processing the audio / video data before playing
  • streaming using protocols that are not supported by the built-in media player

In this blog entry we will show a method to broadcast video (and audio) from an Android phone to a network server, without writing to the file system.

There is one promising method in the MediaRecorder class setOutputFile(FileDescriptor).

We know that in Linux also network sockets have file descriptors. But how could we access the file descriptor of a regular java.net.Socket?

Luckily, ParcelFileDescriptor comes to the rescue, where we can use the fromSocket(Socket) static method to create a ParcelFileDescriptor instance from a Socket object. From this instance, we may now grab the badly needed FileDescriptor.

It all boils down to these few lines (in pseudocode):

String hostname = "your.host.name";
int port = 1234;

Socket socket = new Socket(InetAddress.getByName(hostname), port);

ParcelFileDescriptor pfd = ParcelFileDescriptor.fromSocket(socket);

MediaRecorder recorder = new MediaRecorder();

// Additional MediaRecorder setup (output format ... etc.) omitted

recorder.setOutputFile(pfd.getFileDescriptor());

recorder.prepare();

recorder.start();

Using this concept we created a small proof of concept application (together with an even simpler server), which is able to broadcast videos not limited in length by the available space on the SD Card.

There are a few gotchas, if you want to try this out yourself:

  • The MediaRecorder records either in 3GPP or in MP4 format. This file format consists of atoms, where each atom starts with its size. There are different kinds of atoms in a file, mdat atoms store the actual raw frames of the encoded video and audio. In the Cupcake version Android starts writing out an mdat atom with the encoded frames, but it has to leave the size of the atom empty for obvious reasons. When writing to a seekable file descriptor, it can simply fill in the blanks after the recording, but of course socket file descriptors are not seekable. So the received stream will have to be fixed up after the recording is finished, or the raw video / audio frames have to be processed by the server.
  • For some reason, the MediaRecorder also leaves the header of the file blank, which also has to be handled on the server.
  • High latency connections will cause the video to be choppy. Obviously some buffering is necessary. One method is to use a local mini server on the phone which receives the stream, buffers it, and sends to the remote server as fast as the network allows it. However, if using native code is an option, we can simply create a pipe to receive the data from the MediaRecorder. We will show this method in a future blog entry.

Trackback URL for this post:

http://www.mattakis.com/trackback/30

Comments

very usefull

very usefull material,
thanks

Hi kisg, can you provide its

Hi kisg,

can you provide its full working source please?

really appreciate your help

Thanks for sharing great

Thanks for sharing great technique..

obviously very few people will know abt socket to fd.

thx
suds

Any tips for server side?

Hi

Can You please give some tips for the server side?
What are you using to capture/broadcast the stream?

thx
T

hi i have try to receive and

hi i have try to receive and play by MediaPlayer using this way

but i can't setup and play it.

could you give me some hint for that?

Socket socket = new Socket(serverAddr, PORT); //use tcp to receive
pfd = ParcelFileDescriptor.fromSocket(socket); //set pfd
MP = new MediaPlayer();
MP.setAudioStreamType(AudioManager.STREAM_MUSIC); //create MediaPlayer
MP.setDataSource(pfd.getFileDescriptor()); //set datasource by pfd
MP.prepare();
MP.start();

do i miss to set something?

thx

the same

Hi

i habe the save problem.

do u have got it out now?

thanks

Friend have you find any

Friend have you find any solution ?? plz let me know if you have.

Have u found its working.

Have u found its working, If yes then plz share it. I am get stuck at same point.
Thanks,
Parag Patel

Hi, have anyone of you find

Hi, have anyone of you find the solution to get the mediaplayer play from the filedescriptor/socket?

Thank you!

Good post, but I have one more question...

Hi all,

really useful post... I would like to add one more element. I also need to stream raw data without any form of video encoding... is it possible to do???

Thanks and how about DatagramSocket?

Thanks for this wonderful post. I've been pulling my hair out on this for hours now. Is it possible to do a similar trick with a datagram socket?

Mark

Trying to get this to work

Hi kisg,

I've been trying to get this to work for a few weeks now... Do you have more information/documentation or code I could look at? It would be incredibly helpful if you could post them on the site or send them to me. Thanks.

Pat

Great

Great kisg,

please more details and snipped code, could be a success for all.
Thanks
Stefano

"For some reason, the

"For some reason, the MediaRecorder also leaves the header of the file blank, which also has to be handled on the server."
Any ideas how to handle this?
Is there a posibility to generate the header on the Server?

Awesome! Works flawlessly and

Awesome!
Works flawlessly and smooth!
I didn't even had to fix the stream, as the file generated on the server is working.
My previous attempt at this was to create 10s files and send them.

I m working in same project

I m working in same project .pls can u provide me ur source code.i m struck here.My id: nishantbhatter@gmail.com.
Thnks in Advance.

Repairing the Output

Just to help those having issues, the SDK seems to try to seek to insert the size values of the mdat atom, and also the moov header.

I set the encoder in this example to produce a THREE_GPP file.

In order to play the output THREE_GPP, you're going to need to first create the header in the first 28 bytes prior to the mdat atom (which should all be zeros).

00 00 00 18 66 74 79 70 33 67 70 34 00 00 03 00 33 67 70 34 33 67 70 36 00 02 F1 4D 6D

The 6D is the 'm' first byte in the mdat atom. The four bytes proceeding that need to be modified to include the integer value of the byte in your stream containing the output moov atom (Which should be output upon stopping the recording). As long as this header is correctly set, and the player can locate the moov atom- everything should play back correctly.

Also, the socket method here isn't very flexible- you can perform finer alterations of the packet data to a network (I'm attempting this at the moment for live streaming), by providing it with a local socket, and then connecting to that local socket and processing its output independently (In a thread for instance) for transmission over UDP, RTP, etc..

- Jason

Play incoming stream

Can we do same thing (user local socket) to receive incoming RTP stream which has H326 encodec data?

I have successfully send RTP stream, but when same way going to receive incoming RTP stream, MediaPlayer setDataSource function goes fail. giving me "offset error". I have looked Android MediaPlyer code, its giving error at below point.


status_t MediaPlayerService::Client::setDataSource(int fd, int64_t offset, int64_t length)
{
LOGV("setDataSource fd=%d, offset=%lld, length=%lld", fd, offset, length);
struct stat sb;
int ret = fstat(fd, &sb);
if (ret != 0) {
LOGE("fstat(%d) failed: %d, %s", fd, ret, strerror(errno));
return UNKNOWN_ERROR;
}
LOGV("st_dev = %llu", sb.st_dev);
LOGV("st_mode = %u", sb.st_mode);
LOGV("st_uid = %lu", sb.st_uid);
LOGV("st_gid = %lu", sb.st_gid);
LOGV("st_size = %llu", sb.st_size);
if (offset >= sb.st_size) {
LOGE("offset error");
::close(fd);
return UNKNOWN_ERROR;
}

Can any one help me to understand what I am doing wrong?

Thanks,
Parag Patel

It needs to understand the format.

Parag,

I'm trying to the same thing now- setting the data source to an fd with RTP data on it isn't going to work- since it has no ability to automatically detect the format type. There's no way to set this manually either :( MediaPlayer does support RTSP though, and it should be possible to send it the right SDP information to accomplish this.. I'm amazed nobody else has done this as far as I can tell...

- Jason Thomas.

Source

Hi Parag,

Can you please share the code for sending RTP stream?

Thanks,
Anu

"Just to help those having

"Just to help those having issues, the SDK seems to try to seek to insert the size values of the mdat atom, and also the moov header.

I set the encoder in this example to produce a THREE_GPP file.

In order to play the output THREE_GPP, you're going to need to first create the header in the first 28 bytes prior to the mdat atom (which should all be zeros).

00 00 00 18 66 74 79 70 33 67 70 34 00 00 03 00 33 67 70 34 33 67 70 36 00 02 F1 4D 6D

The 6D is the 'm' first byte in the mdat atom. The four bytes proceeding that need to be modified to include the integer value of the byte in your stream containing the output moov atom (Which should be output upon stopping the recording). As long as this header is correctly set, and the player can locate the moov atom- everything should play back correctly.

Also, the socket method here isn't very flexible- you can perform finer alterations of the packet data to a network (I'm attempting this at the moment for live streaming), by providing it with a local socket, and then connecting to that local socket and processing its output independently (In a thread for instance) for transmission over UDP, RTP, etc..

- Jason"

Thank google (and you, Jason) for this!

I merely wanted to stream the MediaRecorder data immediately to a server for storing a usable file, but was obviously unsuccessful due to missing header information. Upon fixing up the file as you indicated, I got what I wanted.

While it can't be played as a stream due to the moov box data not preceding the mdat box (apparently a requirement for streaming player reception), that is not my current need.

Hi Jason, Can you tell me how

Hi Jason,

Can you tell me how do I go into the 3gp file and edit it in order to make it playable? Should i just take file coming over the stream and write into the file? Also how do i detect the position of the moov atom in the file?

I hope I've understood everything you explained correctly. Correct me if I'm wrong.

-Bhavya

Can you send me code

Hi ! thank you for your solution, but can you send me code for solution

Hi There, Very good tip!

Hi There,

Very good tip! Thanks a lot!
Anyway, how did you append the bytes? I mean, how to get the "stop recording" moment? I've tried a lot of ways to do that but had no success.
Thank you again!

sample code in sipdroid.org

you can get the sample code in VideoCamera.java in Sipdroid.org.

as for playback, it seems that android mediaplayer can not play rtp stream for lacking sdp information. if I am wrong, please correct me.
thanks.

Play the stream

Hi folks,

I think I made some code that works, with some help from this page and looking at the code of the sipdroid. Now I want to play this stream in my computer, but I don't know how. It tried with VLC but I couldn't make it work. I am using ubuntu 9.04. Someone have some tips for me? If you are interested in the code I can paste it here.

Regards,

Roger

Hi Roger, Sorry, don't have

Hi Roger,

Sorry, don't have any tips for you, but would love to see working code for this process.

Thanks!
Scott

Hello!

Hello everyone! I tried to do it too, but had no luck with it... Could someone share his code? I think it would help a lot of people....

Thank in advance!

nice work

you say J2ME can stream video to a server but I dont think it can?

The android code would be most useful but I will try this later, thanks for the great post.

Hi, You can specify

Hi,

You can specify setOutputStream for the RecordControl. Then you can take this stream and send it to a remote server through the network.

Best Regards,
Gergely

There's no

There's no "setOutputStream()" method for "MediaRecorder"

Hi, coudl you guys (blogger

Hi, coudl you guys (blogger or you others) that made it work all the way) so you can see whats being captured on the pc, share the code please?

Im having troubles figuring how to handle the stream at the other end (server)

I got the same troubles in

I got the same troubles in the other end too.

we need to set the format

we need to set the format manually, if it is required then just tell me how to set the format manually as media player is not accepting RTSP

SIP PROTOCOL IN ANDROID

I just started developing a video conferencing application.
Can any body please tell me how to get started using SIP protocol in android.

THANKS IN ADVANCE.

plz contact with me i need ur

plz contact with me i need ur help . I am also working on video conferencing. i Am a student.

any codes can be shared

any codes can be shared please?

how could I write the receiver-side code

Hello , I'm very appreciate this article , it does help me a lot .
now , I can send my video stream from my phone to my desktop PC ,
but How could I receive the video.
I write a simple code as following , but I can't open the received file
[ in Server side ]
FileOutputStream fos;
try {
fos = new FileOutputStream(video);
byte[] data = new byte[1024];

int count =-1;
int times=0;
while( (count = fin.read(data,0,1024) ) !=-1)
{
fos.write(data,0,count);
fos.flush();
}
fos.close();
fin.close();

is there any error?
thank you very much in advance!!!

Excellent post .... but need to know few basics.

Hi there,

I must say that its one excellent post, however i am still struggling with the transporting mechanism of the Video Stream to the server i.e. how would we transport the stream to our servers when we we get the stream on the socket.
I will really appreciate a little help in here as I am about to bang ma head against the wall to search for RTP transport mechanism for video streams in Android.

Regards,
Junaid Anwar Syed.

Can you please tell me how to

Can you please tell me how to edit the headers of the file and how to make the stream playable as it is received at the receiver?

If I have to put the moov box before the mdata box,how do I do it?

What is best solution after a long time

Hello,

This post is so popular and inspired so many developers for a direction. I would like to what is the best solution for streaming from Android to Streaming Server?

Xuggle? ffmpeg??

Thanks,
Subbu

This code does not work on Samsung Galaxy

This code does not work on Android Samsung Galaxy 5700.
Operator start() initiates RuntimeException.
Does anyone knows the way to fix this error?

Server side ? How to play in VLC?

How do i implement the server side in my PC?

How do i play it using VLC?

Any help would be appreciated!!!

Hello kisg i m making an

Hello kisg
i m making an application which enables video conferencing between two android devices. Plz help me out abt how to start . and which class to use. if u have relevent codes plz share them here i will be very thankful to u . PLZ REPLY

Broadcasting video with Android - without writing to local files

Hi sir,

I read the blog. it is great. I tried it, i just create a simple TCP server on the desktop in LAN and try to send video data which is through the ParcelFileDecriptor's output stream, but on server side data could not reach or either reach 28 bytes and all are 0's. I write the code as you written in the blog but server could not get the video data.

Please help.

regards

Kunal

Any advances? Any source available?

Hello everyone! Is there any code available to help us all?
Thanks!

I tried this application and

I tried this application and getting the following error when i do recorder.start() and recording does not start.

E/AuthorDriver( 1961): Command 14 completed with error -1
E/MediaRecorder( 3687): start failed: -1

Complete log

/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:1
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:4
E/audio_input( 1961): unsupported parameter: x-pvmf/media-input-node/cap-config-interface;valtype=key_specific_value
E/audio_input( 1961): VerifyAndSetParameter failed
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:5
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:7
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:11
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:13
E/PVOMXEncNode( 1961): PVMFOMXEncNode-Audio_AMRNB::DoPrepare(): Got Component OMX.SEC.amrenc handle
D/AuthorDriver( 1961): ***TW8282:AuthorDriver::Run()-ac->which:14
I/SMP4FM ( 1961): filename=(null), file=0x58340
I/SMP4FM ( 1961): scmn_mfal_init out
I/SMP4FM ( 1961): use external file pointer
I/SMP4FM ( 1961): ext_fp = 0x58340
I/AudioHardwareALSA( 1961): AudioHardwareALSA::openInputStream - devices = 0x40000 format = 1, channels = 16, sampleRate = 8000
I/AudioHardwareALSA( 1961): AudioHardwareALSA::openInputStream: sampleRate=8000
I/AudioHardwareALSA( 1961): AudioStreamInALSA - input - format = 1, channels = 16, rate = 8000
I/AudioHardwareALSA( 1961): Initialized ALSA CAPTURE device AndroidRecord_Speaker_normal
D/AudioHardwareALSA( 1961): Set CAPTURE PCM format to S16_LE (Signed 16 bit Little Endian)
D/AudioHardwareALSA( 1961): Using 1 channel for CAPTURE.
D/AudioHardwareALSA( 1961): Set CAPTURE sample rate to 8000 HZ
D/AudioHardwareALSA( 1961): Buffer size: 12348
D/AudioHardwareALSA( 1961): Latency: 1543500
I/AudioHardwareALSA( 1961): bufferSize=2240, bytes size of bufferSize=4480
E/AuthorDriver( 1961): Command 14 completed with error -1
E/MediaRecorder( 3687): start failed: -1

I am using Samsung Galaxy 3 i5801. I tested this on both Eclair and Froyo.
Could you please help me on this error ?

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

CAPTCHA
This question is to prevent automated spam submissions.
Image CAPTCHA
Copy the characters (respecting upper/lower case) from the image.