与我们熟知的 MP3 格式一样,AAC 是一种音频编码格式,对比 MP3 格式,AAC 在缩小 30% 的前题下可以提供更好的音质。这篇博客的主要内容就是通过 AudioRecorder 录制 PCM 音频,再通过 MediaCodec 将 PCM 数据硬编码为 AAC 格式的音频。
通常我们使用 MediaCodec 的流程如下:
MediaCodec 的使用流程:
- createEncoderByType/createDecoderByType
- configure
- start
- while(1) {
- dequeueInputBuffer
- queueInputBuffer
- dequeueOutputBuffer
- releaseOutputBuffer
- }
- stop
- release
编解码器一个比较经典的工作原理图如下:
图中的 Client 一般就是我们开发者,解释一下就是:我们从 Codec 中拿到拿到空的 input buffer,然后填充上我们需要进行编码的数据,再输送给 Codec,Codec 对数据进行编解码,编解码完成后,Codec 将处理好的数据放进 output buffer,我们取出后再清空返还给 Codec,形成一个环形结构。可以看作一个生产者-消费者模式。
下面我们的编码流程也基本遵守上面的这个流程。为了便于大家理解,我画个流程图讲一下我整个代码的逻辑:
录音和编码分别在两个线程中进行,两个线程通过一个 ArrayBlockQueue(这是一个线程安全的队列,想了解更多自己动手)队列共享数据,录音线程中的 AudioRecorder 通过 read()将一帧数据 put()到队尾,编码线程中的 MediaCodec 再通过 take()取出队首的一帧数据进行编码。
首先看录音线程,为了使代码更加简洁易懂,我会省略掉一些代码,完整代码会在文末贴出。
/** * 录音线程 */ public class AudioRecorder extends Thread { private AudioRecord mAudioRecord; private boolean isRecording; private int minBufferSize; public AudioRecorder() { isRecording = true; initRecorder(); } @Override public void run() { super.run(); startRecording(); } /** * 初始化录音 */ public void initRecorder(){ minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat); mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, sampleRateInHz, channelConfig, audioFormat, minBufferSize); if (mAudioRecord.getState() != AudioRecord.STATE_INITIALIZED) { isRecording = false; return; } } /** * 开始录音 */ public void startRecording(){ if (mAudioRecord == null){ return; } mAudioRecord.startRecording(); while (isRecording) { //自定义的一个类,用来存储一帧pcm数据,即byte[],下面给出具体定义,很简单 AudioDate audioDate = new AudioDate(); audioDate.buffer = ByteBuffer.allocateDirect(minBufferSize); audioDate.size = mAudioRecord.read(audioDate.buffer, minBufferSize); try { if (queue != null) { queue.put(audioDate); } } catch (InterruptedException e) { e.printStackTrace(); } } release(); } }
录音线程比较简单,主要是先初始化录音器在 initRecorder()中,然后通过 AudioRecorder 的 read 方法,获取到一帧数据,通过 queue.put 放入队尾。
然后是编码线程。
/** * 音频编码线程 */ public class AudioEncorder extends Thread { private MediaCodec mEncorder; private Boolean isEncording = false; private int minBufferSize; private OutputStream mFileStream; public AudioEncorder() { isEncording = true; initEncorder(); } @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) @Override public void run() { super.run(); startEncording(); } /** * 初始化编码器 */ private void initEncorder(){ minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat); try { mEncorder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC); } catch (IOException e) { e.printStackTrace(); } MediaFormat format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRateInHz, channelConfig); format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC); format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC); format.setInteger(MediaFormat.KEY_BIT_RATE, 96000); format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, minBufferSize * 2); mEncorder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE); } /** * 开始编码 */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void startEncording(){ if (mEncorder == null){ return; } mEncorder.start(); try { mFileStream = new FileOutputStream(getSDPath() + "/aac_encode.aac"); MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo(); AudioDate audioDate; while (isEncording) { // 从队列中取出录音的一帧音频数据 audioDate = getAudioDate(); if (audioDate == null) { continue; } // 取出InputBuffer,填充音频数据,然后输送到编码器进行编码 int inputBufferIndex = mEncorder.dequeueInputBuffer(0); if (inputBufferIndex >= 0) { ByteBuffer inputBuffer = mEncorder.getInputBuffer(inputBufferIndex); inputBuffer.clear(); inputBuffer.put(audioDate.buffer); mEncorder.queueInputBuffer(inputBufferIndex, 0, audioDate.size, System.nanoTime(), 0); } // 取出编码好的一帧音频数据,然后给这一帧添加ADTS头 int outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0); while (outputBufferIndex >= 0) { int outBitsSize = mBufferInfo.size; int outPacketSize = outBitsSize + 7; // ADTS头部是7个字节 ByteBuffer outputBuffer = mEncorder.getOutputBuffer(outputBufferIndex); outputBuffer.position(mBufferInfo.offset); outputBuffer.limit(mBufferInfo.offset + outBitsSize); byte[] outData = new byte[outPacketSize]; addADTStoPacket(outData, outPacketSize); outputBuffer.get(outData, 7, outBitsSize); outputBuffer.position(mBufferInfo.offset); mFileStream.write(outData); mEncorder.releaseOutputBuffer(outputBufferIndex, false); outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0); } } release(); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } /** * 添加ADTS头 * @param packet * @param packetLen */ private void addADTStoPacket(byte[] packet, int packetLen) { int profile = 2; //AAC LC int freqIdx = 4; //44100 根据不同的采样率修改这个值 int chanCfg = 2; //CPE packet[0] = (byte) 0xFF; packet[1] = (byte) 0xF9; packet[2] = (byte) (((profile - 1) << 6) + (freqIdx << 2) + (chanCfg >> 2)); packet[3] = (byte) (((chanCfg & 3) << 6) + (packetLen >> 11)); packet[4] = (byte) ((packetLen & 0x7FF) >> 3); packet[5] = (byte) (((packetLen & 7) << 5) + 0x1F); packet[6] = (byte) 0xFC; } }
和录音线程一样,先进行编码器的初始化在 initEncorder()中,这里的参数 MediaFormat 比较关键,针对具体的编码格式有不同的编码格式有不同的参数,其他的编码方式自行参阅官方文档,这里的 AAC 编码器参数我是经过测试可行的。
然后是最关键的一步--编码。在 startEncording()方法中,所有的流程完全对应上面我列出的 MediaCodec 的使用流程,可以对照上面的流程来阅读代码。
编码流程中非常重要的一步就是为每一帧 AAC 音频添加 ADTS 头,和 WAV 格式的音频不同,AAC 为每一帧音频都添加了一个 ADTS 头,使得解码器可以从任意一帧开始解码,有时我们遇到无法编码的 AAC 文件无法播放,可能就是因为我们没有为其添加 ADTS 头。通过上面的 addADTStoPacket()方法便可以未每一帧 AAC 音频添加 ADTS 头了。
至此,我就列出了我认为在 AAC 编码的过程中所需要注意的所有点。在完成这个功能的时候,我也参照了一些别人的做法,以及我自己的一些理解与实践,也遇到了一些令人头大的错误(例如:同样的 PCM 数据,放进队列中再取出来写入文件,就出现了很大的杂音)如有错误的地方,还请留言指正。
下面是完整代码:
package com.example.sisyphus.audiovideolearning; import android.Manifest; import android.app.Activity; import android.content.pm.PackageManager; import android.media.AudioFormat; import android.media.AudioRecord; import android.media.MediaCodec; import android.media.MediaCodecInfo; import android.media.MediaFormat; import android.media.MediaRecorder; import android.os.Build; import android.os.Bundle; import android.os.Environment; import android.support.annotation.Nullable; import android.support.annotation.RequiresApi; import android.support.v4.app.ActivityCompat; import android.view.MotionEvent; import android.view.View; import android.widget.Button; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import java.io.OutputStream; import java.nio.ByteBuffer; import java.util.concurrent.ArrayBlockingQueue; /** * Created by sisyphus on 2018/8/8. * 学习 MediaCodec API,完成音频 AAC 硬编、硬解 * * MediaCodec的使用流程: * - createEncoderByType/createDecoderByType * - configure * - start * - while(1) { * - dequeueInputBuffer * - queueInputBuffer * - dequeueOutputBuffer * - releaseOutputBuffer * } * - stop * - release */public class AACCodecActivity extends Activity { private final int sampleRateInHz = 44100; private final int channelConfig = 1; private final int audioFormat = AudioFormat.ENCODING_PCM_16BIT; private class AudioDate { private ByteBuffer buffer; private int size; } public static String[] MICROPHONE = {Manifest.permission.RECORD_AUDIO}; public static String[] STORAGE = {Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.WRITE_EXTERNAL_STORAGE}; private AudioRecorder mAudioRecorder; private AudioEncorder mAudioEncorder; private ArrayBlockingQueue queue; private Button btnStartRecording; @Override protected void onCreate(@Nullable Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_aac_codec); btnStartRecording = findViewById(R.id.btn_start_aac_encode); queue = new ArrayBlockingQueue<>(1024); mAudioRecorder = new AudioRecorder(); mAudioEncorder = new AudioEncorder(); btnStartRecording.setOnTouchListener(new View.OnTouchListener() { @Override public boolean onTouch(View view, MotionEvent motionEvent) { int action = motionEvent.getAction(); if (action == MotionEvent.ACTION_DOWN) { checkRecordPermission(); btnStartRecording.setText("正在录制"); startRecord(); } else if (action == MotionEvent.ACTION_UP) { btnStartRecording.setText("录音"); stopRecord(); } return false; } }); } private void startRecord() { mAudioRecorder.start(); mAudioEncorder.start(); } private void stopRecord() { mAudioRecorder.stopRecording(); mAudioEncorder.stopEncording(); } /** * 录音线程 */ public class AudioRecorder extends Thread { private AudioRecord mAudioRecord; private boolean isRecording; private int minBufferSize; public AudioRecorder() { isRecording = true; initRecorder(); } @Override public void run() { super.run(); startRecording(); } /** * 初始化录音 */ public void initRecorder(){ minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat); mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, sampleRateInHz, channelConfig, audioFormat, minBufferSize); if (mAudioRecord.getState() != AudioRecord.STATE_INITIALIZED) { isRecording = false; return; } } /** * 释放资源 */ public void release() { if (mAudioRecord != null && mAudioRecord.getState() == AudioRecord.STATE_INITIALIZED) { mAudioRecord.stop(); } } /** * 开始录音 */ public void startRecording(){ if (mAudioRecord == null){ return; } mAudioRecord.startRecording(); while (isRecording) { AudioDate audioDate = new AudioDate(); audioDate.buffer = ByteBuffer.allocateDirect(minBufferSize); audioDate.size = mAudioRecord.read(audioDate.buffer, minBufferSize); try { if (queue != null) { queue.put(audioDate); } } catch (InterruptedException e) { e.printStackTrace(); } } release(); } /** * 结束录音 */ public void stopRecording() { isRecording = false; } } /** * 音频编码线程 */ public class AudioEncorder extends Thread { private MediaCodec mEncorder; private Boolean isEncording = false; private int minBufferSize; private OutputStream mFileStream; public AudioEncorder() { isEncording = true; initEncorder(); } @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) @Override public void run() { super.run(); startEncording(); } /** * 初始化编码器 */ private void initEncorder(){ minBufferSize = AudioRecord.getMinBufferSize(sampleRateInHz, channelConfig, audioFormat); try { mEncorder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC); } catch (IOException e) { e.printStackTrace(); } MediaFormat format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRateInHz, channelConfig); format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC); format.setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC); format.setInteger(MediaFormat.KEY_BIT_RATE, 96000); format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, minBufferSize * 2); mEncorder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE); } /** * 开始编码 */ @RequiresApi(api = Build.VERSION_CODES.LOLLIPOP) public void startEncording(){ if (mEncorder == null){ return; } mEncorder.start(); try { mFileStream = new FileOutputStream(getSDPath() + "/aac_encode.aac"); MediaCodec.BufferInfo mBufferInfo = new MediaCodec.BufferInfo(); AudioDate audioDate; while (isEncording) { // 从队列中取出录音的一帧音频数据 audioDate = getAudioDate(); if (audioDate == null) { continue; } // 取出InputBuffer,填充音频数据,然后输送到编码器进行编码 int inputBufferIndex = mEncorder.dequeueInputBuffer(0); if (inputBufferIndex >= 0) { ByteBuffer inputBuffer = mEncorder.getInputBuffer(inputBufferIndex); inputBuffer.clear(); inputBuffer.put(audioDate.buffer); mEncorder.queueInputBuffer(inputBufferIndex, 0, audioDate.size, System.nanoTime(), 0); } // 取出编码好的一帧音频数据,然后给这一帧添加ADTS头 int outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0); while (outputBufferIndex >= 0) { int outBitsSize = mBufferInfo.size; int outPacketSize = outBitsSize + 7; // ADTS头部是7个字节 ByteBuffer outputBuffer = mEncorder.getOutputBuffer(outputBufferIndex); outputBuffer.position(mBufferInfo.offset); outputBuffer.limit(mBufferInfo.offset + outBitsSize); byte[] outData = new byte[outPacketSize]; addADTStoPacket(outData, outPacketSize); outputBuffer.get(outData, 7, outBitsSize); outputBuffer.position(mBufferInfo.offset); mFileStream.write(outData); mEncorder.releaseOutputBuffer(outputBufferIndex, false); outputBufferIndex = mEncorder.dequeueOutputBuffer(mBufferInfo, 0); } } release(); } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } } /** * 停止编码 */ public void stopEncording() { isEncording = false; } /** * 从队列中取出一帧待编码的音频数据 * @return */ public AudioDate getAudioDate(){ if (queue != null){ try { return queue.take(); } catch (InterruptedException e) { e.printStackTrace(); } } return null; } /** * 添加ADTS头 * @param packet * @param packetLen */ private void addADTStoPacket(byte[] packet, int packetLen) { int profile = 2; //AAC LC int freqIdx = 4; //44100 根据不同的采样率修改这个值 int chanCfg = 2; //CPE packet[0] = (byte) 0xFF; packet[1] = (byte) 0xF9; packet[2] = (byte) (((profile - 1) << 6) + (freqIdx << 2) + (chanCfg >> 2)); packet[3] = (byte) (((chanCfg & 3) << 6) + (packetLen >> 11)); packet[4] = (byte) ((packetLen & 0x7FF) >> 3); packet[5] = (byte) (((packetLen & 7) << 5) + 0x1F); packet[6] = (byte) 0xFC; } /** * 释放资源 */ public void release() { if (mFileStream != null) { try { mFileStream.flush(); mFileStream.close(); } catch (IOException e) { e.printStackTrace(); } } if (mEncorder != null) { mEncorder.stop(); } } } public String getSDPath() { // 判断是否挂载 if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) { return Environment.getExternalStorageDirectory().getAbsolutePath(); } return Environment.getRootDirectory().getAbsolutePath(); } public void checkRecordPermission() { if (ActivityCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) { ActivityCompat.requestPermissions(this, MICROPHONE, 1); return; } if (ActivityCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) { ActivityCompat.requestPermissions(this, STORAGE, 1); return; } } }
欢迎来到这里!
我们正在构建一个小众社区,大家在这里相互信任,以平等 • 自由 • 奔放的价值观进行分享交流。最终,希望大家能够找到与自己志同道合的伙伴,共同成长。
注册 关于