手机网站加速器,2021软件公司排名,网页制作与网站设计论文,wordpress 影响力1.简述与应用范围 ExpPlayer是一个开源的#xff0c;App等级的媒体API#xff0c;它的开源项目包含了library和示例。 ExoPlayer相较于MediaPlayer有很多优点#xff1a; 1. 支持基于http的移动流媒体协议#xff0c;包括DASH#xff0c;HSL#xff0c;Smooth Stream。同… 1.简述与应用范围 ExpPlayer是一个开源的App等级的媒体API它的开源项目包含了library和示例。 ExoPlayer相较于MediaPlayer有很多优点 1. 支持基于http的移动流媒体协议包括DASHHSLSmooth Stream。同时也支持文件流和udp流等。2. 支持更多媒体封装格式包括mp4mp3Webmaacmkvmpeg-ts。3. 支持DRMDigital Right Management 数字版权管理。4. 支持HD高清播放。5. 支持自定义和拓展使用场景。2.上层调用方式 本节说明重点为demo。 简单来说上层调用方式基本为 PlayerActivity - DemoPlayer - ExoPlayer PlayerActivity - RendererBuilder - ExtractorRendererBuilder 类图为 其中PlayerActivity面向UI层一方面控制了播放器DemoPlayer一方面选择了Renderer。 这里的Renderer指定了数据源格式、解码方式和缓冲区大小等。说明这里的缓冲区大小指RollingSampleBuffer的大小不会影响进入播放的速度只会影响缓存数据的最大值 ExoPlayer则是媒体API接口。 DemoPlayer中直接封装了ExoPlayer和相关回调接口负责播放器的逻辑控制和传入SurfaceView等操作而非播放器的内部原理。 这里通过时序图来说明Demo中几个类的调用和封装方式。 3.代码结构 简单来说代码结构是这样 ExoPlayer -ExoPlayerImpl - ExoPlayerImplInternal - TrackRenderer MediaCodecVideoTrackRenderer MediaCodecAudioTrackRenderer - MediaCodecTrackRenderer - SampleSourceTrackRenderer - SampleSource,SampleSourceReader ExtractorSampleSource - DataSource Extractor Loader 这里ExoPlayer为接口。ExoPlayerImpl为实现实现的一些详细步骤在ExoPlayerImplInternal中。后者用Handler消息机制进行异步通信必要时会阻塞。 TrackRenderer是渲染器接口。 MediaCodecTrackRenderer中加入了MediaCodecAndroid硬解码。这里能看出ExoPlayer用的是硬解并且要求4.1以上Android系统。 SampleSourceTrackRenderer中调用了SampleSource,SampleSourceReader接口。SampleSource在这里指的是解封装后的媒体数据。 ExtractorSampleSource相当于一个核心控制器它实现了SampleSource和SampleSourceReader接口。它通过实际的控制线程Loader把从某DataSource即数据源中传过来的原始数据传递给某Extractor来解封装。原始数据解析成SampleSource后储存在RollingSampleBuffer即环形缓冲区中。 MediaCodecTrackRenderer会间接通过ExtractorSampleSource间接从RollingSampleBuffer中读取数据并渲染成画面显示到SurfaceView中。 最后的过程有些复杂流程图如下所示 4.代码原理 1.ExoPlayer - ExoPlayerImpl - ExoPlayerImplInternal 通过以下这段ExoPlayerImpl的构造方法代码可以看出来ExoPlayerImpl中持有一个ExoPlayerImplInternal对象来控制播放器。创建ExoPlayerImplInternal对象时传入了一个eventHandler对象把底层的错误信息和状态改变信息传递给上层处理。 ExoPlayerImpl类中构造方法eventHandler new Handler() {Overridepublic void handleMessage(Message msg) {ExoPlayerImpl.this.handleEvent(msg);}
};
internalPlayer new ExoPlayerImplInternal(eventHandler, playWhenReady, selectedTrackIndices,minBufferMs, minRebufferMs);具体的功能性代码块都在ExoPlayerImplInternal中实现。 状态改变信息和错误信息会通过eventHandler传上来进行处理。 ExoPlayerImpl类// Not private so it can be called from an inner class without going through
// a thunk method.
/* package */ void handleEvent(Message msg) {switch (msg.what) {case ExoPlayerImplInternal.MSG_PREPARED: {System.arraycopy(msg.obj, 0, trackFormats, 0, trackFormats.length);playbackState msg.arg1;for (Listener listener : listeners) {listener.onPlayerStateChanged(playWhenReady, playbackState);}break;}case ExoPlayerImplInternal.MSG_STATE_CHANGED: {playbackState msg.arg1;for (Listener listener : listeners) {listener.onPlayerStateChanged(playWhenReady, playbackState);}break;}case ExoPlayerImplInternal.MSG_SET_PLAY_WHEN_READY_ACK: {pendingPlayWhenReadyAcks--;if (pendingPlayWhenReadyAcks 0) {for (Listener listener : listeners) {listener.onPlayWhenReadyCommitted();}}break;}case ExoPlayerImplInternal.MSG_ERROR: {ExoPlaybackException exception (ExoPlaybackException) msg.obj;for (Listener listener : listeners) {listener.onPlayerError(exception);}break;}}
}这里的listeners是一个CopyOnWriteArrayList里面的对象都是Listener这里用的是一个观察者模式用于给上层监听回调消息。上层即DemoPlayer或是EventLogger都在这里注册或注销监听。 2.ExoPlayerImplInternal - TrackRenderer - SampleSource,SampleSourceReader - ExtractorSampleSource 1ExoPlayerImplInternal中消息机制 ExoPlayerImplInternal类中构造方法internalPlaybackThread new PriorityHandlerThread(getClass().getSimpleName() :Handler,Process.THREAD_PRIORITY_AUDIO);internalPlaybackThread.start();handler new Handler(internalPlaybackThread.getLooper(), this);ExoPlayerImplInternal实现了Handler.Callback接口 ExoPlayerImplInternal类Override
public boolean handleMessage(Message msg) {try {switch (msg.what) {case MSG_PREPARE: {prepareInternal((TrackRenderer[]) msg.obj);return true;}case MSG_INCREMENTAL_PREPARE: {incrementalPrepareInternal();return true;}case MSG_SET_PLAY_WHEN_READY: {setPlayWhenReadyInternal(msg.arg1 ! 0);return true;}case MSG_DO_SOME_WORK: {doSomeWork();return true;}case MSG_SEEK_TO: {seekToInternal(Util.getLong(msg.arg1, msg.arg2));return true;}case MSG_STOP: {stopInternal();return true;}case MSG_RELEASE: {releaseInternal();return true;}case MSG_CUSTOM: {sendMessageInternal(msg.arg1, msg.obj);return true;}case MSG_SET_RENDERER_SELECTED_TRACK: {setRendererSelectedTrackInternal(msg.arg1, msg.arg2);return true;}default:return false;}} catch (ExoPlaybackException e) {Log.e(TAG, Internal track renderer error., e);eventHandler.obtainMessage(MSG_ERROR, e).sendToTarget();stopInternal();return true;} catch (RuntimeException e) {Log.e(TAG, Internal runtime error., e);eventHandler.obtainMessage(MSG_ERROR, new ExoPlaybackException(e, true)).sendToTarget();stopInternal();return true;}
}通过这段代码可以看出来在ExoPlayerImplInternal内部是通过消息来控制播放器逻辑控制TrackRenderer。 2doSomeWork分析及作用 ExoPlayerImplInternal类private void doSomeWork() throws ExoPlaybackException {TraceUtil.beginSection(doSomeWork);long operationStartTimeMs SystemClock.elapsedRealtime();long bufferedPositionUs durationUs ! TrackRenderer.UNKNOWN_TIME_US ? durationUs : Long.MAX_VALUE;boolean allRenderersEnded true;boolean allRenderersReadyOrEnded true;updatePositionUs();// 笔记更新positionUsfor (int i 0; i enabledRenderers.size(); i) {TrackRenderer renderer enabledRenderers.get(i);// TODO: Each renderer should return the maximum delay before which// it wishes to be// invoked again. The minimum of these values should then be used as// the delay before the next// invocation of this method.// 笔记这里调用了renderer的doSomeWork方法并传入了positionUs// elapsedRealtimeUs是个独立的系统时间参考renderer.doSomeWork(positionUs, elapsedRealtimeUs);allRenderersEnded allRenderersEnded renderer.isEnded();// Determine whether the renderer is ready (or ended). If its not,// throw an error thats// preventing the renderer from making progress, if such an error// exists.boolean rendererReadyOrEnded rendererReadyOrEnded(renderer);if (!rendererReadyOrEnded) {renderer.maybeThrowError();}allRenderersReadyOrEnded allRenderersReadyOrEnded rendererReadyOrEnded;if (bufferedPositionUs TrackRenderer.UNKNOWN_TIME_US) {// Weve already encountered a track for which the buffered// position is unknown. Hence the// media buffer position unknown regardless of the buffered// position of this track.} else {long rendererDurationUs renderer.getDurationUs();long rendererBufferedPositionUs renderer.getBufferedPositionUs();if (rendererBufferedPositionUs TrackRenderer.UNKNOWN_TIME_US) {bufferedPositionUs TrackRenderer.UNKNOWN_TIME_US;} else if (rendererBufferedPositionUs TrackRenderer.END_OF_TRACK_US|| (rendererDurationUs ! TrackRenderer.UNKNOWN_TIME_US rendererDurationUs ! TrackRenderer.MATCH_LONGEST_US rendererBufferedPositionUs rendererDurationUs)) {// This track is fully buffered.} else {bufferedPositionUs Math.min(bufferedPositionUs, rendererBufferedPositionUs);}}}// 笔记更新缓冲位置主要用于上层回调this.bufferedPositionUs bufferedPositionUs;// 笔记根据durationUs和positionUs来判断状态和开关渲染器Rendererif (allRenderersEnded (durationUs TrackRenderer.UNKNOWN_TIME_US || durationUs positionUs)) {setState(ExoPlayer.STATE_ENDED);stopRenderers();} else if (state ExoPlayer.STATE_BUFFERING allRenderersReadyOrEnded) {setState(ExoPlayer.STATE_READY);if (playWhenReady) {startRenderers();}} else if (state ExoPlayer.STATE_READY !allRenderersReadyOrEnded) {rebuffering playWhenReady;setState(ExoPlayer.STATE_BUFFERING);stopRenderers();}// 笔记准备再次调用doSomeworkhandler.removeMessages(MSG_DO_SOME_WORK);if ((playWhenReady state ExoPlayer.STATE_READY) || state ExoPlayer.STATE_BUFFERING) {scheduleNextOperation(MSG_DO_SOME_WORK, operationStartTimeMs, RENDERING_INTERVAL_MS);} else if (!enabledRenderers.isEmpty()) {scheduleNextOperation(MSG_DO_SOME_WORK, operationStartTimeMs, IDLE_INTERVAL_MS);}TraceUtil.endSection();
}private void scheduleNextOperation(int operationType, long thisOperationStartTimeMs, long intervalMs) {long nextOperationStartTimeMs thisOperationStartTimeMs intervalMs;long nextOperationDelayMs nextOperationStartTimeMs - SystemClock.elapsedRealtime();if (nextOperationDelayMs 0) {handler.sendEmptyMessage(operationType);} else {handler.sendEmptyMessageDelayed(operationType, nextOperationDelayMs);}
}// 笔记通过上层传入的eventHandler把状态改变信息传递给上层
private void setState(int state) {if (this.state ! state) {this.state state;eventHandler.obtainMessage(MSG_STATE_CHANGED, state, 0).sendToTarget();}
}doSomeWork方法是在播放器执行完prepare后执行的。是在准备动作都完成后具体控制播放器开始渲染画面的方法。 在以上代码中我们可以看出来这里完成的主要动作有 1. 更新positionUs以及elapsedRealtimeUs
2. renderer.doSomeWork
3. 把播放状态回调上层
4. 定时执行下一次doSomeWork3updataPositionUs和renderer.doSomeWork分析 positionUs指的是实际渲染位置。 ExoPlayerImplInternal类private void updatePositionUs() {if (rendererMediaClock ! null enabledRenderers.contains(rendererMediaClockSource) !rendererMediaClockSource.isEnded()) {positionUs rendererMediaClock.getPositionUs();standaloneMediaClock.setPositionUs(positionUs);} else {positionUs standaloneMediaClock.getPositionUs();}elapsedRealtimeUs SystemClock.elapsedRealtime() * 1000;
}通过这段在ExoPlayerImplInternal类中的代码我们看出这有两个分支第一个分支主要是用于有音频的情况下音频时间可以作为整体参考时间来调整positionUs。第二个分支是没有音频的情况下用系统独立时钟作为整体参考时间来调整positionUs。 MediaCodecTrackRenderer类Override
protected void doSomeWork(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackException {// 笔记判断是否应该继续缓冲sourceState continueBufferingSource(positionUs)? (sourceState SOURCE_STATE_NOT_READY ? SOURCE_STATE_READY : sourceState) : SOURCE_STATE_NOT_READY;// 笔记判断解码是否连续如果不连续则重启解码器checkForDiscontinuity(positionUs);if (format null) {// 笔记读取格式readFormat(positionUs);}if (codec null shouldInitCodec()) {// 笔记当有格式无解码器时开启解码器maybeInitCodec();}if (codec ! null) {TraceUtil.beginSection(drainAndFeed);// 笔记如果解码器中可以输出缓冲则会返回true否则返回falsewhile (drainOutputBuffer(positionUs, elapsedRealtimeUs)) {}// 笔记如果解码器还可以输入原始帧则返回true否则返回false第二个参数代表是否首次执行if (feedInputBuffer(positionUs, true)) {while (feedInputBuffer(positionUs, false)) {}}TraceUtil.endSection();}codecCounters.ensureUpdated();
}positionUs传递给了drainOutputBuffer方法和feedInputBuffer方法。用于调整播放时间和获取缓冲帧。 drainOutputBuffer方法调用到了processOutputBuffer方法这里处理缓冲帧。这个方法在MediaCodecTrackRenderer类中是个抽象方法具体实现在MediaCodecVideoTrackRenderer和MediaCodecAudioTrackRenderer类中。 MediaCodecVideoTrackRenderer类// 笔记返回true意味着输出的缓冲帧已经被渲染false意味着尚未被渲染
Override
protected boolean processOutputBuffer(long positionUs, long elapsedRealtimeUs, MediaCodec codec, ByteBuffer buffer,MediaCodec.BufferInfo bufferInfo, int bufferIndex, boolean shouldSkip) {if (shouldSkip) {skipOutputBuffer(codec, bufferIndex);return true;}if (!renderedFirstFrame) {if (Util.SDK_INT 21) {renderOutputBufferV21(codec, bufferIndex, System.nanoTime());} else {renderOutputBuffer(codec, bufferIndex);}return true;}if (getState() ! TrackRenderer.STATE_STARTED) {return false;}// Compute how many microseconds it is until the buffers presentation// time.long elapsedSinceStartOfLoopUs (SystemClock.elapsedRealtime() * 1000) - elapsedRealtimeUs;long earlyUs bufferInfo.presentationTimeUs - positionUs - elapsedSinceStartOfLoopUs;// Compute the buffers desired release time in nanoseconds.long systemTimeNs System.nanoTime();long unadjustedFrameReleaseTimeNs systemTimeNs (earlyUs * 1000);// Apply a timestamp adjustment, if there is one.long adjustedReleaseTimeNs frameReleaseTimeHelper.adjustReleaseTime(bufferInfo.presentationTimeUs,unadjustedFrameReleaseTimeNs);earlyUs (adjustedReleaseTimeNs - systemTimeNs) / 1000;// 笔记以上是通过positionUs实际渲染位置elapsedRealtimeUs独立时钟位置// bufferInfo.presentationTimeUs缓冲帧位置得出缓冲位置和播放位置之间的时间差值。// 笔记如果渲染位置在此缓冲帧位置后面30ms则弃掉此帧if (earlyUs -30000) {// Were more than 30ms late rendering the frame.dropOutputBuffer(codec, bufferIndex);return true;}if (Util.SDK_INT 21) {// 笔记如果系统api在21以上则可以在framework层控制渲染速度// Let the underlying framework time the release.// 笔记如果渲染位置在缓冲帧位置50毫秒之前就return false。否则则渲染。if (earlyUs 50000) {renderOutputBufferV21(codec, bufferIndex, adjustedReleaseTimeNs);return true;}} else {// 笔记如果系统api在21以下我们需要自己控制渲染速度// We need to time the release ourselves.if (earlyUs 30000) {// 笔记如果渲染位置和缓冲帧位置之差在30毫秒和11毫秒之间则推迟至少1毫秒再渲染。// 如果在11毫秒以内则直接渲染。if (earlyUs 11000) {// Were a little too early to render the frame. Sleep until// the frame can be rendered.// Note: The 11ms threshold was chosen fairly arbitrarily.try {// Subtracting 10000 rather than 11000 ensures the sleep// time will be at least 1ms.Thread.sleep((earlyUs - 10000) / 1000);} catch (InterruptedException e) {Thread.currentThread().interrupt();}}renderOutputBuffer(codec, bufferIndex);return true;}}// Were either not playing, or its not time to render the frame yet.// 笔记return false的意思是我们既不播放而且也不渲染这帧。return false;
}在renderOutputBuffer中 codec.releaseOutputBuffer(bufferIndex, true);通过releaseOutputBuffer方法把相关帧播放到surface中。 以上是通过positionUs调整缓冲时间以及播放缓冲帧的代码。 在feedInputBuffer中 result readSource(positionUs, formatHolder, sampleHolder, false);通过readSource调用到了ExtractorSampleSource中的readData方法从rollingBuffer中取到了数据。 这是通过positionUs获取缓冲帧的代码。 通过这些代码可以分析出如果positionUs获取错误的话那么会直接影响到播放流程中从缓冲区获取数据和解码器渲染数据等功能。 3.ExtractorSampleSource - DataSource Extractor Loader 1ExtractingLoadable分析 ExtractingLoadable是一个ExtractorSampleSource中的内部类。它实现了Loadable接口。Loadable接口应用于Loader后者是一个异步线程。在这里主要用于从DataSource数据源中获取数据放进RollingSampleBuffer即缓冲区中。 /*** Loads the media stream and extracts sample data from it.*/
private static class ExtractingLoadable implements Loadable {private final Uri uri;private final DataSource dataSource;private final ExtractorHolder extractorHolder;private final Allocator allocator;private final int requestedBufferSize;private final PositionHolder positionHolder;private volatile boolean loadCanceled;private boolean pendingExtractorSeek;public ExtractingLoadable(Uri uri, DataSource dataSource, ExtractorHolder extractorHolder, Allocator allocator,int requestedBufferSize, long position) {this.uri Assertions.checkNotNull(uri);this.dataSource Assertions.checkNotNull(dataSource);this.extractorHolder Assertions.checkNotNull(extractorHolder);this.allocator Assertions.checkNotNull(allocator);this.requestedBufferSize requestedBufferSize;positionHolder new PositionHolder();positionHolder.position position;pendingExtractorSeek true;}// 笔记用于控制线程的关闭Overridepublic void cancelLoad() {loadCanceled true;}Overridepublic boolean isLoadCanceled() {return loadCanceled;}Overridepublic void load() throws IOException, InterruptedException {int result Extractor.RESULT_CONTINUE;while (result Extractor.RESULT_CONTINUE !loadCanceled) {ExtractorInput input null;try {long position positionHolder.position;// 笔记开打数据源这里C.LENGTH_UNBOUNDED值为-1long length dataSource.open(new DataSpec(uri, position, C.LENGTH_UNBOUNDED, null));if (length ! C.LENGTH_UNBOUNDED) {length position;}// 笔记这里的ExtractorInput是一个对于数据源、读取位置、读取长度的封装// 用于向Extractor输入数据input new DefaultExtractorInput(dataSource, position, length);// 笔记通过数据选择正确的Extractor即文件封装拆解器Extractor extractor extractorHolder.selectExtractor(input);if (pendingExtractorSeek) {extractor.seek();pendingExtractorSeek false;}// 笔记这个循环用于从Extractor中不断读取数据放进RollingSampleBuffer中while (result Extractor.RESULT_CONTINUE !loadCanceled) {allocator.blockWhileTotalBytesAllocatedExceeds(requestedBufferSize);result extractor.read(input, positionHolder);// TODO: Implement throttling to stop us from buffering// data too often.}} finally {if (result Extractor.RESULT_SEEK) {result Extractor.RESULT_CONTINUE;} else if (input ! null) {positionHolder.position input.getPosition();}// 笔记关闭数据源dataSource.close();}}}}我们可以看出线程中进行的主要动作是 1. dataSource.open即打开数据源
2. Extractor extractor extractorHolder.selectExtractor(input)选择正确的文件封装拆解器
3. result extractor.read(input, positionHolder)从数据源中读取数据
4. dataSource.close关闭数据源2ExtractorHolder分析 ExtractorHolder也是一个ExtractorSampleSource中的内部类。它主要负责持有Extractor。 ExtractorHolder类public Extractor selectExtractor(ExtractorInput input)throws UnrecognizedInputFormatException, IOException, InterruptedException {if (extractor ! null) {return extractor;}for (Extractor extractor : extractors) {try {// 笔记一旦识别到正确的解析器则会返回trueif (extractor.sniff(input)) {this.extractor extractor;break;}} catch (EOFException e) {// Do nothing.}input.resetPeekPosition();}if (extractor null) {throw new UnrecognizedInputFormatException(extractors);}// 笔记这里调用了extractor.init即初始化extractor.init(extractorOutput);return extractor;}3Extractor分析 Extractor是个接口表示文件封装解析器。里面主要有四个方法 void init(ExtractorOutput output);boolean sniff(ExtractorInput input) throws IOException, InterruptedException;int read(ExtractorInput input, PositionHolder seekPosition) throws IOException, InterruptedException;void seek();read方法是阻塞的。每次调用read只会获取一小部分数据。 同时这里定义了三个read方法的特殊返回值 RESULT_CONTINUE 0; //表示需要继续读取数据RESULT_SEEK 1; //表示需要重新定位数据RESULT_END_OF_INPUT C.RESULT_END_OF_INPUT; //表示已经读取结束通过Extractor的实现类我们可以找到当调用read方法时都会调到trackOutput.sampleData方法。这个方法表示输出解封装后的帧。具体就是把解封装的帧存入RollingSampleBuffer中在TrackOutput的实现类DefaultTrackOutput中的如下代码可以印证这一点 Override
public void sampleData(ParsableByteArray buffer, int length) {rollingBuffer.appendData(buffer, length);
}具体的文件解封装这里不做细节分析。 4.其他 5.相关性补充 ijkplayer中Android部分 ijkplayer是bilibili推出的同时支持ios和Android硬解和软解的开源播放器框架。其中在Android代码中硬解部分应用了ExoPlayer软解部分应用了ffmepg和sdl。 ijkplayer的demo中调用方式是这样的 VideoActivity - IjkVideoView - IMediaPlayer - AbstractMediaPlayer AbstractMediaPlayer - IjkExoMediaPlayer - DemoPlayer - ExoPlayer AbstractMediaPlayer - IjkMediaPlayer - ijkplayer_jni.c - ijkplayer.c - Ff_ffplayer.c 原文链接http://blog.csdn.net/henryjax/article/details/51313051
