OpenMAX

From DaphneWiki

Jump to: navigation, search

Contents

OpenMAX IL

Core

Dynamically loads/unloads components.

Sets up tunnels between two components.

Accesses component's methods.

API is probably the same across platforms.

Component

Individual blocks of functionality, including "sources, sinks, codecs, filters, splitters, mixers, or any other data operator."

Types of components:
Source: component with single output port
Sink: component with single input port
Host: run entirely on host processor

Component provider defines functionality of given component.

Operate on four types of data: audio, video, image, and other (time data for sync).

Provides access to standard set of component functions via component handle.

Must have at least one port to conform to official standard.

Data communication calls to components are non-blocking.

Ports

"Data communication to and from a component is conducted through interfaces called ports. Ports represent both the connection for components to the data stream and the buffers needed to maintain the connection. Users may send data to components through input ports or receive data through output ports. Similarly, a communication tunnel between two components can be established by connecting the output port of one component to a similarly formatted input port of another component."

Ports must support callbacks to the IL client.

Communication

  1. Non-tunneled communication: buffer exchange between IL client and component.
  2. Tunneled communication: direct buffer exchange between two components.
  3. Proprietary: we don't want to deal with this nonsense.

Profiles

  1. Base: does not need to implement tunneling
  2. Interop: must implement tunneling

States

All components start in the "unloaded" state and must be "loaded" via a call to the core. All other state transitions may then be achieved by communicating directly with the component. On error, the component automatically reverts to the "unloaded" state.

  1. WAIT FOR RESOURCES: waiting for initialization to finish
  2. IDLE: component has all static resources but is not processing data ("stopped")
  3. EXECUTING: pending reception of buffers to process data, will make callbacks
  4. PAUSED: won't process any data, can resume where it left off (idle cannot)


IL client

Your application, some third-party framework, or OpenMAX AL.

Always communicates to components via core.

hello_video OMX API calls

Here is a sequence of all the OMX API calls that the hello_video sample program makes (from raspberry pi raspbian sample image).

  1. OMX_Init()
  2. OMX_GetHandle(pHandleDecode, "OMX.broadcom.video_decode", pAppData, &callbacks)
  3. OMX_GetComponentVersion(pHandleDecode, pszNameArray, pCompVersion, pSpecVersion, pUID) name: "OMX.broadcom.video_decode:22", version: 0's, spec version: 1.1.2, uid same as name
  4. OMX_GetParameter(pHandleDecode, OMX_IndexParamAudioInit, &pPorts) (nPorts returned as 0)
  5. OMX_GetParameter(pHandleDecode, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 2) (ports.nStartPortNumber returned 130)
  6. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 130, NULL)
  7. (waits for event callback to fire in other thread to indicate that port has been disabled)
  8. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 131, NULL)
  9. (waits for event callback to fire in other thread to indicate that port has been disabled)
  10. OMX_GetHandle(pHandleRender, "OMX.broadcom.video_render", pAppData, &callbacks)
  11. OMX_GetComponentVersion(pHandleRender, ...)
  12. OMX_GetParameter(pHandleRender, OMX_IndexParamAudioInit, &pPorts) (nPorts returned as 0)
  13. OMX_GetParameter(pHandleRender, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 1) (ports.nStartPortNumber returned 90)
  14. OMX_GetHandle(pHandleClock, "OMX.broadcom.clock", ...)
  15. OMX_GetParameter(pHandleClock, OMX_IndexParamOtherInit, ...) (nPorts returned as 6) (ports.nStartPortNumber returned UNKNOWN, FIND OUT)
  16. OMX_SetParameter(pHandleClock?, OMX_IndexConfigTimeClockState, &cState)
  17. OMX_GetHandle(pHandleSched, "OMX.broadcom.video_scheduler", ...)
  18. OMX_GetParameter(pHandleSched, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 2) (ports.nStartPortNumber returned 10)
  19. OMX_GetParameter(pHandleSched, OMX_IndexParamAudioInit, &pPorts) (nPorts returned 1) (ports.nStartPortNumber returned 12)
  20. OMX_GetState(pHandleClock, &state) (returns state of OMX_StateLoaded)
  21. OMX_SendCommand(pHandleClock, OMX_CommandStateSet, OMX_StateIdle, NULL)
  22. (waits for command to complete)
  23. OMX_SendCommand(pHandleClock, OMX_CommandPortDisable, 80, NULL)
  24. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 12, NULL)
  25. (waits for both previous commands to finish)
  26. OMX_GetParameter(pHandleClock, OMX_IndexParamNumAvailableStreams, &param) (returned an error because source port does not use streams)
  27. OMX_SetupTunnel(pHandleClock, 80, pHandleSched, 12);
  28. OMX_SendCommand(pHandleClock, OMX_CommandPortEnable, 80, NULL);
  29. OMX_SendCommand(pHandleSched, OMX_CommandPortEnable, 12, NULL);
  30. OMX_GetState(pHandleSched, &state) (returns OMX_StateLoaded)
  31. (wait for enable tunnel command to complete for pHandeSched)
  32. OMX_SendCommand(pHandleSched, OMX_CommandStateSet, OMX_StateIdle, NULL)
  33. (wait for state change command to complete)
  34. (wait for enable tunnel to complete for pHandleClock)
  35. OMX_SendCommand(pHandleClock, OMX_CommandStateSet, OMX_StateExecute, NULL)
  36. OMX_SendCommand(pHandleDecode, OMX_CommandStateSet, OMX_StateIdle, NULL)
  37. OMX_SetParameter(pHandleDecode, OMX_IndexParamVideoPortFormat, &format)
  38. OMX_GetParameter(pHandleDecode, OMX_IndexParamPortDefinition, &portdef) (index is 130)
  39. (portdef.bEnabled == OMX_FALSE, portdef.nBufferCountActual == 20, portdef.nBufferSize == 81920, portdef.nBufferAlignment == 16, portdef.eDir == OMX_DirInput, portdef.eDomain == OMX_PortDomainVideo)
  40. OMX_GetState(pHandleDecode, &state) (state is OMX_StateIdle)
  41. OMX_SendCommand(pHandleDecode, OMX_CommandPortEnable, 130, NULL)
  42. (allocates buffer of 81920 bytes in size)
  43. OMX_UseBuffer(pHandleDecode, ?, 130, NULL, 81920, bufThatWasAllocated)
  44. (this allocation of buffer followed by the OMX_UseBuffer call loops 20 times)
  45. OMX_SendCommand(pHandleDecode, OMX_CommandStateSet, OMX_StateExecuting, NULL)
  46. (get a unique buffer.. somehow.. lol)
  47. (fread video stream into buffer)
  48. (set buf->nFilledLen to the bytes we read, buf->nOffset = 0, buf->nFlags = OMX_BUFFERFLAG_STARTTIME)
  49. OMX_EmptyThisBuffer(pHandleDecode, buf) where buf points to a OMX_BUFFERHEADTYPE struct
  50. Keep doing the former starting with getting the unique buffer until we notice that port 131 has received a OMX_EventPortSettingsChanged event
  51. begin one-time step
  52. Set up tunnel from port 131 (decode) to port 10 (scheduler)
  53. change video scheduler's state to executing
  54. Set up tunnel from port 11 (scheduler) to port 90 (render)
  55. change render state to executing
  56. end of the one-time-step
  57. continue looping (getting a unique buffer, etc) until fread gets end-of-file
  58. OMX_EmptyThisBuffer(pHandleDecode, buf) where buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN | OMX_BUFFERFLAG_EOS and nFilledLen is 0
  59. Wait for event from video render, where the event is OMX_EventBufferFlag of OMX_BUFFERFLAG_EOS
  60. flush tunnels so that decoder can disable its input port by doing the following:
  61. OMX_SendCommand(pHandleDecode, OMX_CommandFlush, 131, NULL)
  62. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 10, NULL)
  63. wait for two flush events to be received
  64. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 11, NULL)
  65. OMX_SendCommand(pHandleRender, OMX_CommandFlush, 90, NULL)
  66. wait for two flush events to be received
  67. OMX_SendCommand(pHandleClock, OMX_CommandFlush, 80, NULL)
  68. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 12, NULL)
  69. wait for two flush events to be received
  70. OMX_GetParameter(pHandleDecode, OMX_IndexParamPortDefinition, &portdef) (where port is 130)
  71. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 130, NULL)
  72. OMX_FreeBuffer(pHandleDecode, 130, ?) <-- loop 20 times for each buffer
  73. wait for port disable to finish
  74. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 131, NULL)
  75. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 10, NULL)
  76. wait for port disable to finish
  77. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 11, NULL)
  78. OMX_SendCommand(pHandleRender, OMX_CommandPortDisable, 90, NULL)
  79. wait for port disable to finish
  80. OMX_SendCommand(pHandleClock, OMX_CommandPortDisable, 80, NULL)
  81. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 12, NULL)
  82. wait for port disable to finish
  83. OMX_SetupTunnel(pHandleDecode, 131, NULL, 0)
  84. OMX_SetupTunnel(pHandleSched, 10, NULL, 0)
  85. OMX_SetupTunnel(pHandleSched, 11, NULL, 0)
  86. OMX_SetupTunnel(pHandleRender, 90, NULL, 0)
  87. OMX_SetupTunnel(pHandleClock, 80, NULL, 0)
  88. OMX_SetupTunnel(pHandleSched, 12, NULL, 0)
  89. change state of every handle to OMX_StateIdle (in random order for some unexplained reason)
  90. change state of every handle to OMX_StateLoaded (in random order for some unexplained reason)
  91. OMX_FreeHandle(pHandleDecode)
  92. OMX_FreeHandle(pHandleRender)
  93. OMX_FreeHandle(pHandleClock)
  94. OMX_FreeHandle(pHandleSched)
  95. OMX_Deinit()
Personal tools