本篇乃針對 Audio Unit Hosting Guide for iOS，摘錄個人閱讀後的心得整理。
基本概念
Audio Unit 在 iOS Audio Frame Work 屬於底層的應用，其階層關係可參考下圖。
Audio Unit 提供了快速、模組化的音頻處理，但因為使用方式較複雜，Apple建議只有當具有下列需求時，才需要使用 Audio Unit
- Simultaneous audio I/O (input and output) with low latency, such as for a VoIP (Voice over Internet Protocol) application
- Responsive playback of synthesized sounds, such as for musical games or synthesized musical instruments
- Use of a specific audio unit feature such as acoustic echo cancelation, mixing, or tonal equalization
- A processing-chain architecture that lets you assemble audio processing modules into flexible networks. This is the only audio API in iOS offering this capability.
Audio Units共可區分為四大類，並可細分為七種，可參考下表
參考下圖，Audio Unit 內部結構分為兩部分，分別稱作 Scope 與 Element，其中 scope 分為三種，分別是 input scope, output scope, global scope，而 element 則是 input scope 或 output scope 內的一部分。
下圖是一個 I/O type 的 Audio Unit，其輸入為麥克風，其輸出為喇叭。這便是一個最簡單的Audio Unit使用範例。
● The input element is element 1 (mnemonic device: the letter “I” of the word “Input” has an appearance similar to the number 1)
● The output element is element 0 (mnemonic device: the letter “O” of the word “Output” has an appearance similar to the number 0)
API使用摘要
1. Audio Unit 初始化 ，有兩種初始化方式：
Method a. 直接使用 AudioUnit 進行初始化，
Method b. 透過 AudioGraph 初始化 AudioUnit。
由於大部分的使用情形都會用到 Audio Graph，所以建議還是透過 AudioGraph 使用AudioUnit，以下分別介紹兩種方式。
Method A
//Creating an audio component description to identify an audio unit
AudioComponentDescription ioUnitDescription;
ioUnitDescription.componentType = kAudioUnitType_Output;
ioUnitDescription.componentSubType = kAudioUnitSubType_RemoteIO;
ioUnitDescription.componentManufacturer = kAudioUnitManufacturer_Apple;
ioUnitDescription.componentFlags = 0;
ioUnitDescription.componentFlagsMask = 0;
//Obtaining an audio unit instance using the audio unit API
AudioComponent foundIoUnitReference = AudioComponentFindNext (
NULL,
&ioUnitDescription
);
AudioUnit ioUnitInstance;
AudioComponentInstanceNew (
foundIoUnitReference,
&ioUnitInstance
);
Method B
// Obtaining an audio unit instance using the audio processing graph API
// Declare and instantiate an audio processing graph
AUGraph processingGraph;
NewAUGraph (&processingGraph);
// Add an audio unit node to the graph, then instantiate the audio unit
AUNode ioNode;
AUGraphAddNode (
processingGraph,
&ioUnitDescription,
&ioNode
);
AUGraphOpen (processingGraph); // indirectly performs audio unit instantiation
// Obtain a reference to the newly-instantiated I/O unit
AudioUnit ioUnit;
AUGraphNodeInfo (
processingGraph,
ioNode,
NULL,
&ioUnit
);
2. 設定屬性，可以使用下列 API
AudioUnitGetPropertyInfo, AudioUnitSetProperty, AudioUnitGetProperty, AudioUnitAddPropertyListener, AudioUnitRemovePropertyListenerWithUserData
使用範例：
UInt32 busCount = 2;
OSStatus result = AudioUnitSetProperty (
mixerUnit,
kAudioUnitProperty_ElementCount, // the property key
kAudioUnitScope_Input, // the scope to set the property on
0, // the element to set the property on
&busCount, // the property value
sizeof (busCount)
);
常用的 property 如下：
kAudioOutputUnitProperty_EnableIO // 預設 input is disabled
kAudioUnitProperty_ElementCount
kAudioUnitProperty_MaximumFramesPerSlice// 預設是 1024，當鎖屏時要設定為 4096
kAudioUnitProperty_StreamFormat
3. 設定參數，可以使用下列 API
AudioUnitSetParameter, AudioUnitGetParameter
使用範例：
OSStatus result = AudioUnitSetParameter (
mixerUnit,
kMultiChannelMixerParam_Enable,
kAudioUnitScope_Input,
inputBus,
isOnValue,
0
);
4. 設定 Audio Graph 時，有幾個重點需要注意，如下
- AUGraph 的所有命令都會被放在一個 todo list 內，直到呼叫 AUGraphUpdate，這些命令才會真的作用。
- 當呼叫 AUGraphStart 或 AUGraphStop時，會呼叫對應的 Audio Unit 的 AudioOutputUnitStart or AudioOutputUnitStop
- 可以使用不同的 Audio Unit 分別處理聲音。例如 spec 中的 Figure 1-5 便同時使用了 3 種 Audio UnitiPod Equalizer, Multichannel Mixer unit, Remote I/O unit
- 使用 AudioGraph 時，盡可能不要改變 sample rate
- 必須在某些點，自行設定 audio format。
- 系統也會在硬體輸入或輸出時，自行設定format，(uncompressed, in linear PCM format, and interleaved.) 最好是整個 audio graph 都用同一種 format
5. 設定 Audio Unit Render Callback Function
需注意效能問題。
you must not take locks, allocate memory, access the file system or a network connection, or otherwise perform time-consuming tasks in the body of a render callback function.
Callback Function 原型如下：
static OSStatus MyAURenderCallback (
void *inRefCon,
AudioUnitRenderActionFlags *ioActionFlags,
const AudioTimeStamp *inTimeStamp,
UInt32 inBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData
) { /* callback body */ }
若要輸出靜音，則要設定 ioActionFlags 與 ioData 如下
*ioActionFlags |= kAudioUnitRenderAction_OutputIsSilence;
and memset ioData to 0;
6. 有兩種方式可用來設定Audio Format
- 使用 AudioStreamBasicDescription(ASBD)
- 或是使用 CAStreamBasicDescription，此方式可以多設定三個參數。
7. 設定Audio Fomart時的注意事項
- Whether the stream is for I/O (SetCanonical) or for audio processing (SetAUCanonical)
- How many channels you want the stream format to represent
- Whether you want the stream format interleaved or noninterleaved
需注意以下兩點
AudioUnitSampleType and AudioSampleType are not the same. In iOS, the former is 8.24 fixed point and the latter is SInt16.
kAudioFormatFlagsCanonical 與 kAudioFormatFlagsAudioUnitCanonical 的不同
kAudioFormatFlagsCanonical = kAudioFormatFlagIsFloat | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked,
kAudioFormatFlagsAudioUnitCanonical = kAudioFormatFlagIsFloat | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked | kAudioFormatFlagIsNonInterleaved,
8. 設定 Audio Graph 時，有以下預設的 Design Pattern 可供選擇
a. I/O Pass Through
b. I/O Without a Render Callback Function
c. I/O with a Render Callback Function
d. Output-Only with a Render Callback Function
e. Other Audio Unit Hosting Design Patterns
9. 無論選擇了哪一種design pattern，撰寫程式時仍需要依循以下步驟:
- Configure your audio session.
- Specify audio units.
- Create an audio processing graph, then obtain the audio units.
- Configure the audio units.
- Connect the audio unit nodes.
- Provide a user interface.
- Initialize and then start the audio processing graph.
10. 若需要即時的處理，可以縮短IOBufferDuration
There’s one other hardware characteristic you may want to configure: audio hardware I/O buffer duration. The default duration is about 23 ms at a 44.1 kHz sample rate, equivalent to a slice size of 1,024 samples. If I/O latency is critical in your app, you can request a smaller duration, down to about 0.005 ms (equivalent to 256 samples),
設定範例如下：
self.ioBufferDuration = 0.005;
[mySession setPreferredIOBufferDuration: ioBufferDuration
error: &audioSessionError];
11. 摘錄 Core Audio Glossary 的對於Frame與Slice的名詞解釋
frame
In Core Audio, a set of samples that contains one sample from each channel in an audio data stream. In the most common case, the samples in a frame are time-coincident—that is, sampled at the same moment. For example, in a stereo stream each frame contains one sample from the left channel and a time-coincident sample from the right channel. More generally, the various channels in a stream, and therefore in a frame, may be from unrelated sources and may have originated at unrelated times. In video, a single image in a series that constitutes a movie. See also packet.slice
The number of frames requested and processed during one rendering cycle of an audio unit. See also frame.
程式實作範例
範例一：
摘錄自Audio Mixer
1 AUGraph processingGraph; 2 3 NewAUGraph (&processingGraph); 4 5 // I/O unit 6 AudioComponentDescription iOUnitDescription; 7 iOUnitDescription.componentType = kAudioUnitType_Output; 8 iOUnitDescription.componentSubType = kAudioUnitSubType_RemoteIO; 9 iOUnitDescription.componentManufacturer = 10 kAudioUnitManufacturer_Apple; 11 iOUnitDescription.componentFlags = 0; 12 iOUnitDescription.componentFlagsMask = 0; 13 14 // Multichannel mixer unit 15 AudioComponentDescription MixerUnitDescription; 16 MixerUnitDescription.componentType = kAudioUnitType_Mixer; 17 MixerUnitDescription.componentSubType = 18 kAudioUnitSubType_MultiChannelMixer; 19 MixerUnitDescription.componentManufacturer = 20 kAudioUnitManufacturer_Apple; 21 MixerUnitDescription.componentFlags = 0; 22 MixerUnitDescription.componentFlagsMask = 0; 23 24 AUNode iONode; // node for I/O unit 25 AUNode mixerNode; // node for Multichannel Mixer unit 26 27 // Add the nodes to the audio processing graph 28 result = AUGraphAddNode ( 29 processingGraph, 30 &iOUnitDescription, 31 &iONode); 32 33 if (noErr != result) {[self printErrorMessage: @"AUGraphNewNode failed 34 for I/O unit" withStatus: result]; return;} 35 36 37 result = AUGraphAddNode ( 38 processingGraph, 39 &MixerUnitDescription, 40 &mixerNode 41 ); 42 43 result = AUGraphOpen (processingGraph); 44 45 result = AUGraphNodeInfo ( 46 processingGraph, 47 mixerNode, 48 NULL, 49 &mixerUnit 50 ); 51 52 result = AudioUnitSetProperty ( 53 mixerUnit, 54 kAudioUnitProperty_ElementCount, 55 kAudioUnitScope_Input, 56 0, 57 &busCount, 58 sizeof (busCount) 59 ); 60 61 result = AudioUnitSetProperty ( 62 mixerUnit, 63 kAudioUnitProperty_MaximumFramesPerSlice, 64 kAudioUnitScope_Global, 65 0, 66 &maximumFramesPerSlice, 67 sizeof (maximumFramesPerSlice) 68 ); 69 70 result = AUGraphSetNodeInputCallback ( 71 processingGraph, 72 mixerNode, 73 busNumber, 74 &inputCallbackStruct 75 ); 76 77 result = AudioUnitSetProperty ( 78 mixerUnit, 79 kAudioUnitProperty_StreamFormat, 80 kAudioUnitScope_Input, 81 guitarBus, 82 &stereoStreamFormat, 83 sizeof (stereoStreamFormat) 84 ); 85 86 result = AudioUnitSetProperty ( 87 mixerUnit, 88 kAudioUnitProperty_StreamFormat, 89 kAudioUnitScope_Input, 90 beatsBus, 91 &monoStreamFormat, 92 sizeof (monoStreamFormat) 93 ); 94 95 result = AudioUnitSetProperty ( 96 mixerUnit, 97 kAudioUnitProperty_SampleRate, 98 kAudioUnitScope_Output, 99 0, 100 &graphSampleRate, 101 sizeof (graphSampleRate) 102 ); 103 104 result = AUGraphConnectNodeInput ( 105 processingGraph, 106 mixerNode, // source node 107 0, // source node output bus number 108 iONode, // destination node 109 0 // desintation node input bus number 110 ); 111 112 result = AUGraphInitialize (processingGraph); 113 114 AUGraphStart (processingGraph);
參考資料：
- AudioUnitHostingGuideForiOS.pdf
- Core Audio Data Types Reference
- Audio Unit Properties Reference
- Audio Unit Component Services Reference
- Audio Unit Processing Graph Services Reference
