一、AAC 格式
參考維基百科，AAC 共有以下七種格式- audio/aac,
- audio/aacp,
- audio/3gpp,
- audio/3gpp2,
- audio/mp4,
- audio/MP4A-LATM,
- audio/mpeg4-generic
其中某些 AAC 格式是 Apple 支援的，此部分可以直接使用 Apple Audio Queue Services進行播放，若是Apple不支援的格式，則可以使用 FFMPEG 先解開成 PCM，再使用 Apple Audio Queue Services 播放。
那麼如何判斷是否是 Apple 所支援的格式呢？簡言之，只要AudioFileOpenURL能夠打得開的檔案便可認為 Apple支援。以下提供一個簡單的範例。
註：使用 Apple Audio Queue Services 播放檔案時，也可參考上述用法，直接取得正確的 AudioStreamBasicDescription。
註：使用 Apple Audio Queue Services 播放檔案時，也可參考上述用法，直接取得正確的 AudioStreamBasicDescription。
- (void) PrintFileStreamBasicDescription:(NSString *) filePath{
OSStatus status;
UInt32 size;
AudioFileID audioFile;
AudioStreamBasicDescription dataFormat;
CFURLRef URL = (CFURLRef)[NSURL fileURLWithPath:filePath];
status=AudioFileOpenURL(URL, kAudioFileReadPermission, kAudioFileAAC_ADTSType, &audioFile);
if (status != noErr) {
NSLog(@"*** Error *** PlayAudio - play:Path: could not open audio file. Path given was: %@", filePath);
return ;
}
else {
NSLog(@"*** OK *** : %@", filePath);
}
size = sizeof(dataFormat);
AudioFileGetProperty(audioFile, 0, &size, &dataFormat);
if(size>0){
NSLog(@"mFormatID=%d", (signed int)dataFormat.mFormatID);
NSLog(@"mFormatFlags=%d", (signed int)dataFormat.mFormatFlags);
NSLog(@"mSampleRate=%ld", (signed long int)dataFormat.mSampleRate);
NSLog(@"mBitsPerChannel=%d", (signed int)dataFormat.mBitsPerChannel);
NSLog(@"mBytesPerFrame=%d", (signed int)dataFormat.mBytesPerFrame);
NSLog(@"mBytesPerPacket=%d", (signed int)dataFormat.mBytesPerPacket);
NSLog(@"mChannelsPerFrame=%d", (signed int)dataFormat.mChannelsPerFrame);
NSLog(@"mFramesPerPacket=%d", (signed int)dataFormat.mFramesPerPacket);
NSLog(@"mReserved=%d", (signed int)dataFormat.mReserved);
}
AudioFileClose(audioFile);
}以下舉出兩個AAC例子
- APPLE 支援的AAC
- http://mm2.pcslab.com/mm/7h800.mp4
- APPLE 不支援的AAC
- http://download.wavetlan.com/SVV/Media/HTTP/AAC_12khz_Mono_5.aac
二、Apple Audio Queue Service
若是 Apple 支援的格式，可以直接開啟檔案後使用 Audio Service進行播放。
但因為我的用途是播放 RTSP 所傳送的 AAC 檔案，因此需要將接收到的 AAC 放入 Audio Queue Service。
基本的 Audio Queue 播放音樂的用法，參考 AudioQueueProgrammingGuide，摘錄如下：
- Define a custom structure to manage state, format, and path information.
- Write an audio queue callback function to perform the actual playback.
- Write code to determine a good size for the audio queue buffers.
- Open an audio file for playback and determine its audio data format.
- Create a playback audio queue and configure it for playback.
- Allocate and enqueue audio queue buffers. Tell the audio queue to start playing. When done, the playback callback tells the audio queue to stop.
- Dispose of the audio queue. Release resources.
實作舉例：
// 1. 定義資料結構
#define AUDIO_BUFFER_SECONDS 1
#define AUDIO_BUFFER_QUANTITY 3
#define NUM_BUFFERS 3
AudioStreamBasicDescription audioFormat;
AudioQueueRef mQueue;
AudioQueueBufferRef mBuffers[NUM_BUFFERS];
// 2. 實作對應的 call back 函數，HandleOutputBuffer()，用來填充欲播放的音樂
void HandleOutputBuffer (
void *aqData,
AudioQueueRef inAQ,
AudioQueueBufferRef inBuffer
){
PlayAudio* player=(PlayAudio*)aqData;
[player putAVPacketsIntoAudioQueue:inBuffer];
}
// 3. 設定此Stream相關的屬性，此處可由 ffmpeg 的 AVCodecContext 資料結構取得。
audioFormat.mFormatID = kAudioFormatMPEG4AAC;
audioFormat.mFormatFlags = kMPEG4Object_AAC_Main;
audioFormat.mSampleRate = pAudioCodecCtx->sample_rate;
audioFormat.mFramesPerPacket = pAudioCodecCtx->frame_size;
audioFormat.mChannelsPerFrame = pAudioCodecCtx->channels;
// 4. 建立一個 Audio Queue (mQueue)，並設定對應的 call back 函數為 HandleOutputBuffer()
AudioQueueNewOutput(&audioFormat, HandleOutputBuffer, self, NULL, NULL, 0, &mQueue);
// 5. 配置 3 個 Buffer 給 Audio Queue
for (i = 0; i < AUDIO_BUFFER_QUANTITY; i++) {
AudioQueueAllocateBufferWithPacketDescriptions(mQueue, pAudioCodecCtx->bit_rate * AUDIO_BUFFER_SECONDS / 8,
pAudioCodecCtx->sample_rate * AUDIO_BUFFER_SECONDS / pAudioCodecCtx->frame_size + 1, &mBuffers[i]);
}
// 6.設置音量
Float32 gain=1.0;
AudioQueueSetParameter(mQueue, kAudioQueueParam_Volume, gain);
// 7. 開始播放音樂，系統會自動從已配置的3個buffer內讀取資料，並進行播放
// 若 buffer 內的資料已播完，便會調用 HandleOutputBuffer 再次填充音樂buffer，進行播放。
AudioQueueStart(mQueue, nil);
// 8. putAVPacketsIntoAudioQueue 函數負責填充資料，
// 若是要播放Apple所支援的AAC，則step3所設定的audioFormat便維持不變
// 若是要由ffmpeg解開為PCM，則可在此處實作，並改變step3所設定的audioFormat
三、FFMPEG 將 AAC 轉成 PCM
此處介紹的便是上述 putAVPacketsIntoAudioQueue()的部分實作。對於Apple支援的AAC，此處只需要複製記憶體即可，因此這邊只介紹如何使用 FFMPEG 播放 Apple不支援的AAC的做法，如下：
1. 使用 FFMPEG 讀取 AVPacket
av_read_frame(pFormatCtx, & AudioPacket)2. 將 AVPacket 解開為 AVFrame
avcodec_decode_audio4(pAudioCodecCtx, pAudioFrame, &gotFrame, &AudioPacket);3. 將 AVFrame 複製至 Audio Queue 對應的 buffer
此處需注意的是ffmpeg 所解開的檔案，其bitsPerSample 可能為32,16, 8 bits，若其bitsPerSample 不同，則需要進行轉換，以下的例子便需要將 AV_SAMPLE_FMT_FLTP (U32) 轉換至 AV_SAMPLE_FMT_S16 (S16)，才能夠正常播放。否則播放時便會有許多雜音出現。
實作舉例：
-(UInt32)putAVPacketsIntoAudioQueue:(AudioQueueBufferRef)audioQueueBuffer{
AudioTimeStamp bufferStartTime={0};
AVPacket AudioPacket={0};
int gotFrame = 0;
static int vSlienceCount=0;
AudioQueueBufferRef buffer=audioQueueBuffer;
av_init_packet(&AudioPacket);
buffer->mAudioDataByteSize = 0;
buffer->mPacketDescriptionCount = 0;
if(mIsRunning==false)
{
return 0 ;
}
// TODO: remove debug log
NSLog(@"get 1 from apQueue: %d", [audioPacketQueue count]);
// If no data, we put silence audio
// If AudioQueue buffer is empty, AudioQueue will stop.
if([audioPacketQueue count]==0)
{
int err, vSilenceDataSize = 1024*4;
if(vSlienceCount>10)
{
// Stop fill silence, since the data may be eof or error happen
//[self Stop:false];
mIsRunning = false;
return 0;
}
vSlienceCount++;
NSLog(@"Put Silence -- Need adjust circular buffer");
@synchronized(self)
{
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mStartOffset = buffer->mAudioDataByteSize;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mDataByteSize = vSilenceDataSize;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mVariableFramesInPacket = 1;
buffer->mAudioDataByteSize += vSilenceDataSize;
buffer->mPacketDescriptionCount++;
}
if ((err = AudioQueueEnqueueBuffer(mQueue,
buffer,
0,
NULL)))
{
NSLog(@"Error enqueuing audio buffer: %d", err);
}
return 1;
}
vSlienceCount = 0;
// while (([audioPacketQueue count]>0) && (buffer->mPacketDescriptionCount < buffer->mPacketDescriptionCapacity))
if(buffer->mPacketDescriptionCount < buffer->mPacketDescriptionCapacity)
{
//NSLog(@"aqueue:%d", [audioPacketQueue count]);
[audioPacketQueue getAVPacket: &AudioPacket];
#if DECODE_AUDIO_BY_FFMPEG == 1 // decode by FFmpeg
{
uint8_t *pktData=NULL;
int pktSize;
int len=0;
AVCodecContext *pAudioCodecCtx = aCodecCtx;
AVFrame *pAVFrame1 = pAudioFrame;
pktData=AudioPacket.data;
pktSize=AudioPacket.size;
while(pktSize>0) {
avcodec_get_frame_defaults(pAVFrame1);
@synchronized(self)
{
len = avcodec_decode_audio4(pAudioCodecCtx, pAVFrame1, &gotFrame, &AudioPacket);
}
if(len>0) {
int outCount=0;
int data_size = av_samples_get_buffer_size(NULL, pAudioCodecCtx->channels,
pAVFrame1->nb_samples,pAudioCodecCtx->sample_fmt, 0);
if (buffer->mAudioDataBytesCapacity - buffer->mAudioDataByteSize >= data_size/2) {
@synchronized(self)
{
if(pAudioCodecCtx->sample_fmt==AV_SAMPLE_FMT_FLTP){
int in_samples = pAVFrame1->nb_samples;
// if (buffer->mPacketDescriptionCount == 0)
{
bufferStartTime.mSampleTime = LastStartTime+in_samples;
bufferStartTime.mFlags = kAudioTimeStampSampleTimeValid;
LastStartTime = bufferStartTime.mSampleTime;
}
#if 1
uint8_t pTemp[8][data_size/2];
uint8_t *pOut = (uint8_t *)&pTemp;
outCount = swr_convert(pSwrCtx,
(uint8_t **)(&pOut),
in_samples,
(const uint8_t **)pAVFrame1->extended_data,
in_samples);
#else
// We can use av_samples_alloc() and av_freep() for sample buffer
// But use a static array may be efficience
uint8_t *pOut=NULL;
int out_linesize=0;
av_samples_alloc(&pOut,
&out_linesize,
pAudioFrame->channels,
in_samples,
AV_SAMPLE_FMT_S16,
0
);
outCount = swr_convert(pSwrCtx,
(uint8_t **)&pOut,
in_samples,
(const uint8_t **)pAudioFrame->extended_data,
in_samples);
// TODO: need free pOut
#endif
if(outCount<0 data-blogger-escaped-buffer-="" data-blogger-escaped-fail="" data-blogger-escaped-memcpy="" data-blogger-escaped-nslog="" data-blogger-escaped-swr_convert="" data-blogger-escaped-uint8_t="">mAudioData + buffer->mAudioDataByteSize, pOut, data_size);
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mStartOffset = buffer->mAudioDataByteSize;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mDataByteSize = data_size/2;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mVariableFramesInPacket
= 1;
}
};
buffer->mAudioDataByteSize += data_size/2;
buffer->mPacketDescriptionCount++;
}
gotFrame = 0;
}
pktSize-=len;
pktData+=len;
}
}
#else
if (buffer->mAudioDataBytesCapacity - buffer->mAudioDataByteSize >= AudioPacket.size) {
// if (buffer->mPacketDescriptionCount == 0)
// {
// bufferStartTime.mSampleTime = LastStartTime+pAudioFrame->nb_samples;
// bufferStartTime.mFlags = kAudioTimeStampSampleTimeValid;
// LastStartTime = bufferStartTime.mSampleTime;
// }
memcpy((uint8_t *)buffer->mAudioData + buffer->mAudioDataByteSize, AudioPacket.data, AudioPacket.size);
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mStartOffset = buffer->mAudioDataByteSize;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mDataByteSize = AudioPacket.size;
buffer->mPacketDescriptions[buffer->mPacketDescriptionCount].mVariableFramesInPacket = aCodecCtx->frame_size;
buffer->mAudioDataByteSize += AudioPacket.size;
buffer->mPacketDescriptionCount++;
}
#endif
[audioPacketQueue freeAVPacket:&AudioPacket];
}
if (buffer->mPacketDescriptionCount > 0) {
int err;
#if 0 // CBR
if ((err = AudioQueueEnqueueBuffer(mQueue,
buffer,
0,
NULL)))
#else // VBR
if ((err = AudioQueueEnqueueBufferWithParameters(mQueue,
buffer,
0,
NULL,
0,
0,
0,
NULL,
&bufferStartTime,
NULL)))
#endif
{
NSLog(@"Error enqueuing audio buffer: %d", err);
}
}
return 0;
}
註：以上述的 AAC_12khz_Mono_5.aac 為例，雖然Apple無法自動解開此檔案，但若是自行讀出此aac檔案的每個frame(例如使用ffmpeg的 avformat_read_frame())，將ADTS header拿掉，設定正確的 AudioStreamBasicDescription 之後，再丟給 Apple Audio Queue，還是可以正確播放出聲音的。
實際可運作的程式碼可參考 https://github.com/alb423/FFmpegAudioPlayer/
參考資料：
- AAC 維基百科
- 使用 Audio Queue Services 播放音樂。
- Michael Tyson's circular buffer implementation。
- Wave Header 介紹
- AAC 範例檔案
- 使用 libav 的 resample 的作法
