Android10随处现场直播插图

责任编辑为看雪高峰论坛杰出该文

看雪高峰论坛译者ID:猫熊吃鱼

直接原因

玩着杀虫剂,忽然雅典娜给我来音频了,不过我已经开始玩游戏呀。我省服甜蜜蜜是不是能NDS!

Android10随处现场直播插图1

后郭闻潮我重大胜利了,但我丧失了两个雅典娜。心中很迷惘,只好我想出了两个设想。假如每天雅典娜给我音频,我都是接了。陪她骂人,跳舞给她听,那能很难杨瑞麟了,对,要说的。

流程

首先看一下Android音频现场直播的流程(图片网上百度的):

Android10随处现场直播插图2

能看出来,获取数据的源头其实是在摄像头哪里。至于后面的各种流程都能无视了。那么是说,只要把摄像头的数据替换掉,那直接成功了!

Android10随处现场直播插图3

实战

首先考虑的事情,是是不是把摄像头的数据替换。

设想一:直接在驱动层拦截数据,看了内核的代码放弃;

设想二:修改Camera的代码,看了java代码放弃;

设想三:修改设想二的设想,还是修改java吧。

代码片段

假如想实现上面的设想,步是写代码把摄像头的数据先获取到。

关键的类

SufaceView

这个View能直接重内存或者DMA等硬件接口获取所得的图像数据,是个非常重要的绘图容器,所以,开发相机应用一般都是使用它。

重写的方法:

SurfaceHolder

能把它看成是surface的控制器,用来操纵surface。处理它的Canvas上画的效果和动画,控制表面,大小,像素等等。

SurfaceTexture

能看做Surface和Texture的组合,是将图形生产者的数据送到Texture,然后是由应用程序自己来处理。

surfaceChanged(SurfaceHolderholderformatwidthheight){}surfaceCreated(SurfaceHolderholder){}surfaceDestroyed(SurfaceHolderholder) {}

具体片段代码:

SurfaceHolder holder = mSurfaceView.getHolder();holder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);holder.addCallback(newSurfaceHolder.Callback() {@OverridepublicvoidsurfaceDestroyed(SurfaceHolder arg0){// TODO Auto-generated method stubif(mCamera !=null) {mCamera.stopPreview();mSurfaceView =null;mSurfaceHolder =null;}}@OverridepublicvoidsurfaceCreated(SurfaceHolder arg0){// TODO Auto-generated method stubtry{if(mCamera !=null) {mCamera.setPreviewDisplay(arg0);mSurfaceHolder = arg0;}}catch(IOException exception) {Log.e(TAG,“Error setting up preview display”, exception);}}@OverridepublicvoidsurfaceChanged(SurfaceHolder arg0,intarg1,intarg2,intarg3){// TODO Auto-generated method stubif(mCamera ==null)return;//设置参数Camera.Parameters parameters = mCamera.getParameters();parameters.setPreviewSize(640,480);parameters.setPictureSize(640,480);mCamera.setParameters(parameters);try{mCamera.startPreview();mSurfaceHolder = arg0;}catch(Exception e) {Log.e(TAG,“could not start preview”, e);mCamera.release();mCamera =null;}}});}

这个时候能实现实时显示摄像头的数据了。

Camera.java源码分析

privateclassEventHandlerextendsHandler{privateCamera mCamera;publicEventHandler(Camera c, Looper looper){super(looper);mCamera = c;}@OverridepublicvoidhandleMessage(Message msg){switch(msg.what) {caseCAMERA_MSG_SHUTTER:if(mShutterCallback !=null) {mShutterCallback.onShutter();}return;caseCAMERA_MSG_RAW_IMAGE:if(mRawImageCallback !=null) {mRawImageCallback.onPictureTaken((byte[])msg.obj, mCamera);}return;caseCAMERA_MSG_COMPRESSED_IMAGE:if(mJpegCallback !=null) {mJpegCallback.onPictureTaken((byte[])msg.obj, mCamera);}return;caseCAMERA_MSG_PREVIEW_FRAME:PreviewCallback pCb = mPreviewCallback;if(pCb !=null) {if(mOneShot) {// Clear the callback variable before the callback// in case the app calls setPreviewCallback from// the callback functionmPreviewCallback =null;}elseif(!mWithBuffer) {// Were faking the camera preview mode to prevent// the app from being flooded with preview frames.// Set to oneshot mode again.setHasPreviewCallback(true,false);}pCb.onPreviewFrame((byte[])msg.obj, mCamera);}return;caseCAMERA_MSG_POSTVIEW_FRAME:if(mPostviewCallback !=null) {mPostviewCallback.onPictureTaken((byte[])msg.obj, mCamera);}return;caseCAMERA_MSG_FOCUS:AutoFocusCallback cb =null;synchronized(mAutoFocusCallbackLock) {cb = mAutoFocusCallback;}if(cb !=null) {booleansuccess = msg.arg1 ==0?false:true;cb.onAutoFocus(success, mCamera);}return;caseCAMERA_MSG_ZOOM:if(mZoomListener !=null) {mZoomListener.onZoomChange(msg.arg1, msg.arg2 !=0, mCamera);}return;caseCAMERA_MSG_PREVIEW_METADATA:if(mFaceListener !=null) {mFaceListener.onFaceDetection((Face[])msg.obj, mCamera);}return;caseCAMERA_MSG_ERROR :Log.e(TAG,“Error “+ msg.arg1);if(mErrorCallback !=null) {mErrorCallback.onError(msg.arg1, mCamera);}return;caseCAMERA_MSG_FOCUS_MOVE:if(mAutoFocusMoveCallback !=null) {mAutoFocusMoveCallback.onAutoFocusMoving(msg.arg1 ==0?false:true, mCamera);}return;default:Log.e(TAG,“Unknown message type “+ msg.what);return;}}}

这里经过修改源码打印分析,每天拍照或者录像的时候都会走。

CAMERA_MSG_RAW_IMAGE

CAMERA_MSG_COMPRESSED_IMAGE

CAMERA_MSG_POSTVIEW_FRAME

三个事件

其中的:

caseCAMERA_MSG_COMPRESSED_IMAGE:if(mJpegCallback !=null) {mJpegCallback.onPictureTaken((byte[])msg.obj, mCamera);}return;

(byte[])msg.obj是图片数据:

publicinterfacePreviewCallback{/*** Called as preview frames are displayed. This callback is invoked* on the event thread {@linkopen(int)} was called from.**

If using the {@linkandroid.graphics.ImageFormatYV12} format,* refer to the equations in {@linkCamera.ParameterssetPreviewFormat}* for the arrangement of the pixel data in the preview callback* buffers.**@paramdata the contents of the preview frame in the format defined* by {@linkandroid.graphics.ImageFormat}, which can be queried* with {@linkandroid.hardware.Camera.ParametersgetPreviewFormat()}.* If {@linkandroid.hardware.Camera.ParameterssetPreviewFormat(int)}* is never called, the default will be the YCbCr_420_SP* (NV21) format.*@paramcamera the Camera service object.*/voidonPreviewFrame(byte[] data, Camera camera);};

这里还有两个核心的native函数。

public native final void setPreviewTexture(SurfaceTexture surfaceTexture) throws IOException;private native final void setPreviewDisplay(Surface surface) throws IOException;

总结:直接修改上面分析的函数应该能了。

是不是实现音频播放?

1. ffmepg去播放音频推数据

2. Android原生支持

MediaPlayer

结合SurfaceView进行播放。其中通过SurfaceView显示音频的画面,通过MediaPlayer来设置播放参数、并控制音频的播放操作。

然后百度了两个demo 实现通过SurfaceView显示音频的画面。

把demo 整合到系统源码

Android10随处现场直播插图4

这里我使用原生自带的MediaPlayer 和 自定义实现的两个Player 其实效果差不多 然后把这些类组成一下 VirCamera。

数据格式的问题

调数据yuv数据和存储为yuv420p。

importandroid.graphics.ImageFormat;importandroid.graphics.Rect;importandroid.graphics.YuvImage;importandroid.media.Image;importandroid.text.Spanned;importandroid.util.Log;importcom.android.internal.logging.nano.MetricsProto;importjava.io.ByteArrayOutputStream;importjava.io.FileOutputStream;importjava.io.IOException;importjava.nio.ByteBuffer;publicclassImageUtil{publicstaticfinalintCOLOR_FormatI420 =1;publicstaticfinalintCOLOR_FormatNV21 =2;publicstaticfinalintNV21 =2;privatestaticfinalString TAG =“ImageUtil”;publicstaticfinalintYUV420P =0;publicstaticfinalintYUV420SP =1;privatestaticbooleanisImageFormatSupported(Image image){intformat = image.getFormat();if(format ==17|| format ==35|| format ==842094169) {returntrue;}returnfalse;}publicstaticvoiddumpFile(String str,byte[] bArr){try{FileOutputStream fileOutputStream =newFileOutputStream(str);try{fileOutputStream.write(bArr);fileOutputStream.close();}catch(IOException e) {thrownewRuntimeException(“failed writing data to file “+ str, e);}}catch(IOException e2) {thrownewRuntimeException(“Unable to create output file “+ str, e2);}}publicstaticvoidcompressToJpeg(String str, Image image){try{FileOutputStream fileOutputStream =newFileOutputStream(str);Rect cropRect = image.getCropRect();newYuvImage(getDataFromImage(image,2),17, cropRect.width(), cropRect.height(), (int[])null).compressToJpeg(cropRect,100, fileOutputStream);}catch(IOException e) {thrownewRuntimeException(“Unable to create output file “+ str, e);}}publicstaticbyte[] yuvToCompressedJPEG(byte[] bArr, Rect rect) {ByteArrayOutputStream byteArrayOutputStream =newByteArrayOutputStream();newYuvImage(bArr,17, rect.width(), rect.height(), (int[])null).compressToJpeg(rect,100, byteArrayOutputStream);returnbyteArrayOutputStream.toByteArray();}publicstaticbyte[] getDataFromImage(Image image,inti) {Rect rect;inti2;inti3 = i;inti4 =2;inti5 =1;if(i3 !=1&& i3 !=2) {thrownewIllegalArgumentException(“only support COLOR_FormatI420 and COLOR_FormatNV21”);}elseif(isImageFormatSupported(image)) {Rect cropRect = image.getCropRect();intformat = image.getFormat();intwidth = cropRect.width();intheight = cropRect.height();Image.Plane[] planes = image.getPlanes();inti6 = width * height;byte[] bArr =newbyte[((ImageFormat.getBitsPerPixel(format) * i6) /8)];inti7 =0;byte[] bArr2 =newbyte[planes[0].getRowStride()];inti8 =1;inti9 =0;inti10 =0;while(i9 < planes.length) {switch(i9) {case0:i8 = i5;i10 = i7;break;case1:if(i3 != i5) {if(i3 == i4) {i10 = i6 +1;i8 = i4;break;}}else{i8 = i5;i10 = i6;break;}break;case2:if(i3 != i5) {if(i3 == i4) {i8 = i4;i10 = i6;break;}}else{i10 = (int) (((double) i6) *1.25d);i8 = i5;break;}break;}ByteBuffer buffer = planes[i9].getBuffer();introwStride = planes[i9].getRowStride();intpixelStride = planes[i9].getPixelStride();inti11 = i9 ==0? i7 : i5;inti12 = width >> i11;inti13 = height >> i11;inti14 = width;buffer.position(((cropRect.top >> i11) * rowStride) + ((cropRect.left >> i11) * pixelStride));inti15 =0;while(i15 < i13) {if(pixelStride ==1&& i8 ==1) {buffer.get(bArr, i10, i12);i10 += i12;rect = cropRect;i2 = i12;}else{i2 = ((i12 –1) * pixelStride) +1;rect = cropRect;buffer.get(bArr2,0, i2);inti16 = i10;for(inti17 =0; i17 < i12; i17++) {bArr[i16] = bArr2[i17 * pixelStride];i16 += i8;}i10 = i16;}if(i15 < i13 –1) {buffer.position((buffer.position() + rowStride) – i2);}i15++;cropRect = rect;}Log.v(TAG,“Finished reading data from plane “+ i9);i9++;i5 =1;width = i14;i3 = i;i4 =2;i7 =0;}returnbArr;}else{thrownewRuntimeException(“cant convert Image to byte array, format “+ image.getFormat());}}publicstaticbyte[] getBytesFromImageAsType(Image image,inti) {Image.Plane[] planeArr;Image.Plane[] planeArr2;try{Image.Plane[] planes = image.getPlanes();intwidth = image.getWidth();intheight = image.getHeight();inti2 = width * height;byte[] bArr =newbyte[((ImageFormat.getBitsPerPixel(35) * i2) /8)];byte[] bArr2 =newbyte[(i2 /4)];byte[] bArr3 =newbyte[(i2 /4)];inti3 =0;inti4 =0;inti5 =0;inti6 =0;while(i3 < planes.length) {intpixelStride = planes[i3].getPixelStride();introwStride = planes[i3].getRowStride();ByteBuffer buffer = planes[i3].getBuffer();byte[] bArr4 =newbyte[buffer.capacity()];buffer.get(bArr4);if(i3 ==0) {inti7 = i4;inti8 =0;for(inti9 =0; i9 < height; i9++) {System.arraycopy(bArr4, i8, bArr, i7, width);i8 += rowStride;i7 += width;}planeArr = planes;i4 = i7;}elseif(i3 ==1) {inti10 = i5;inti11 =0;for(inti12 =0; i12 < height /2; i12++) {inti13 =0;while(i13 < width /2) {bArr2[i10] = bArr4[i11];i11 += pixelStride;i13++;i10++;}if(pixelStride ==2) {i11 += rowStride – width;}elseif(pixelStride ==1) {i11 += rowStride – (width /2);}}planeArr = planes;i5 = i10;}elseif(i3 ==2) {inti14 = i6;inti15 =0;inti16 =0;while(i15 < height /2) {inti17 = i16;inti18 =0;while(true) {planeArr2 = planes;if(i18 >= width /2) {break;}bArr3[i14] = bArr4[i17];i17 += pixelStride;i18++;i14++;planes = planeArr2;}if(pixelStride ==2) {i17 += rowStride – width;}elseif(pixelStride ==1) {i17 += rowStride – (width /2);}i15++;i16 = i17;planes = planeArr2;}planeArr = planes;i6 = i14;}else{planeArr = planes;}i3++;planes = planeArr;}switch(i) {case0:System.arraycopy(bArr2,0, bArr, i4, bArr2.length);System.arraycopy(bArr3,0, bArr, i4 + bArr2.length, bArr3.length);break;case1:for(inti19 =0; i19 < bArr3.length; i19++) {inti20 = i4 +1;bArr[i4] = bArr2[i19];i4 = i20 +1;bArr[i20] = bArr3[i19];}break;case2:for(inti21 =0; i21 < bArr3.length; i21++) {inti22 = i4 +1;bArr[i4] = bArr3[i21];i4 = i22 +1;bArr[i22] = bArr2[i21];}break;}returnbArr;}catch(Exception e) {if(image ==null) {returnnull;}image.close();returnnull;}}publicstaticint[] decodeYUV420SP(byte[] bArr,inti,inti2) {inti3 = i;inti4 = i2;inti5 = i3 * i;int[] iArr =newint[i5];inti6 =0;inti7 =0;while(i6 < i4) {inti8 = ((i6 >>1) * i3) + i5;inti9 =0;inti10 =0;inti11 = i7;inti12 =0;while(i12 < i3) {inti13 = (bArr[i11] &255) –16;if(i13 <0) {i13 =0;}if((i12 &1) ==0) {inti14 = i8 +1;inti15 = i14 +1;inti16 = (bArr[i14] &255) –128;i9 = (bArr[i8] &255) –128;i8 = i15;i10 = i16;}inti17 =1192* i13;inti18 = (1634* i9) + i17;inti19 = (i17 – (MetricsProto.MetricsEvent.FIELD_CONTEXT * i9)) – (400* i10);inti20 = i17 + (2066* i10);if(i18 <0) {i18 =0;}elseif(i18 >262143) {i18 =262143;}if(i19 <0) {i19 =0;}elseif(i19 >262143) {i19 =262143;}if(i20 <0) {i20 =0;}elseif(i20 >262143) {i20 =262143;}iArr[i11] = –16777216| ((i18 <<6) & Spanned.SPAN_PRIORITY) | ((i19 >>2) &65280) | ((i20 >>10) &255);i11++;}i6++;i7 = i11;}returniArr;}}

那么现在是见神奇的时候:

Camera(int cameraId) {mShutterCallback =null;mRawImageCallback =null;mJpegCallback =null;mPreviewCallback =null;mPostviewCallback =null;mUsingPreviewAllocation =false;mZoomListener =null;Looper looper;if((looper = Looper.myLooper()) !=null) {mEventHandler =newEventHandler(this, looper);}elseif((looper = Looper.getMainLooper()) !=null) {mEventHandler =newEventHandler(this, looper);}else{mEventHandler =null;}StringpackageName = ActivityThread.currentPackageName();native_setup(newWeakReference(this), cameraId, packageName);//开启虚拟摄像头VirCamera.getInstance().open();}

caseCAMERA_MSG_POSTVIEW_FRAME:if(mPostviewCallback !=null) {VirCamera.getInstance().onPictureTaken((byte[])msg.obj, mCamera);}return;caseCAMERA_MSG_COMPRESSED_IMAGE:if(mJpegCallback !=null) {VirCamera.getInstance().onPictureTaken((byte[])msg.obj, mCamera);}return;

后编译 开始测试,

发现我的雅典娜一直没给我音频,只好换了两个现场直播平台试试。

Android10随处现场直播插图5

嘻嘻应该是成功,坐等雅典娜音频。

Android10随处现场直播插图6

Android10随处现场直播插图7

– End –

Android10随处现场直播插图8

看雪ID:猫熊吃鱼

https://bbs.pediy.com/user-home-830790.htm

*责任编辑由看雪高峰论坛 猫熊吃鱼  原创,转载请注明来自看雪社区。

往期推荐

Unicorn Trace还原Ollvm算法!《Android高级研修班》2021年6月班开始招生!

WPS 2016远程代码执行漏洞(栈溢出)分析及利用

一道算法题

2020MRCTF(Re)-Hard-to-go(WP)

A64dbg尝鲜——实战某加固so CrackMe

深度揭密高通4/5G移动基带消息系统和状态机

Android10随处现场直播插图9

公众号ID:ikanxue

官方微博:看雪安全

商务合作:wsc@kanxue.com

Android10随处现场直播插图10

球分享

Android10随处现场直播插图10

球点赞

Android10随处现场直播插图10

球在看

Android10随处现场直播插图11

点击阅读原文,了解更多!

作者 nasiapp

在线客服
官方客服
我们将24小时内回复。
12:01
您好,有任何疑问请与我们联系!

选择聊天工具: