Unit Testing Custom View in Android

Standard

Preface

In this short article, I want to propose a method on unit testing an Android View.

Anybody who ever wrote UI tests knows it’s quite slow (both in terms of implementation and in terms of execution) and prone to errors and number of problems (especially running on CI). Robolectric improves the situation quite a bit but still has its own problems.

I want to propose an alternative solution, which, while not being applicable everywhere, might provide a faster, better and more stable option for UI testing.

Just give me the code

If you would rather just play with code here’s the repo that contains all the code used in this article. 

High level overview

So what is the approach exactly? 

The main idea is that we don’t need to check if our UI is displayed correctly (most of the time) what we really need is to make sure that we’ve set proper variables or called proper functions with proper arguments.

What does it mean? It means that if we can make sure that our code called:

view.setBackgroundColor(R.color.red)

we don’t need to check that our view became red since we assume that the framework works correctly. And well, if it doesn’t, it is nothing you can fix directly anyway, you can only work around it.

The solution I want to present utilises custom Android view and Mockk, which allows us to achieve exactly what we want – ability to check that methods were called, without dragging the whole Android framework to tests with us. Note that the technique is applicable to Activity, but, since you can’t create an instance of Activity manually, you still will need either Robolectric or instrumentation to provide you with that, which removes much of the benefits of the approach.

Custom view

I will start with the presentation of the code since there is not a lot of it:

import android.content.Context
import android.util.AttributeSet
import android.view.LayoutInflater
import android.widget.LinearLayout
import kotlinx.android.synthetic.main.custom_view.view.*
class CustomView @JvmOverloads constructor(
context: Context, attrs: AttributeSet? = null, defStyleAttr: Int = 0
) : LinearLayout(context, attrs, defStyleAttr) {
init {
LayoutInflater
.from(context)
.inflate(R.layout.custom_view, this)
}
fun initialize() {
orientation = VERTICAL
likeButton.setOnClickListener {
likes.text = (likes.text.toString().toInt() + 1).toString()
}
}
fun update(model: CustomViewModel) {
fact.text = model.text
likes.text = model.numberOfLikes.toString()
}
}
view raw CustomView.kt hosted with ❤ by GitHub

And layout looks like:

<?xml version="1.0" encoding="utf-8"?>
<merge xmlns:android="http://schemas.android.com/apk/res/android&quot;
xmlns:tools="http://schemas.android.com/tools&quot;
tools:orientation="vertical"
tools:layout_width="match_parent"
tools:layout_height="match_parent"
tools:parentTag="com.syllogismobile.unittestingview.CustomView">
<TextView android:id="@+id/fact"
android:layout_width="match_parent"
android:layout_height="0dp"
android:layout_gravity="center_horizontal|top"
android:textSize="@dimen/default_text_size"
android:layout_margin="@dimen/standard_margin"
android:layout_weight="8"
android:ellipsize="end"
tools:text="A cat's jaw has only up and down motion; it does not have any lateral, side to side motion, like dogs and humans."/>
<TextView android:id="@+id/likes"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:gravity="start|center_vertical"
android:textSize="@dimen/default_text_size"
android:textColor="@android:color/holo_red_light"
android:layout_margin="@dimen/standard_margin"
android:layout_weight="1"
tools:text="1"/>
<Button android:id="@+id/likeButton"
android:layout_width="match_parent"
android:layout_height="@dimen/default_button_height"
android:layout_margin="@dimen/standard_margin"
android:layout_gravity="bottom"
android:text="@string/like" />
</merge>
view raw custom_view.xml hosted with ❤ by GitHub

Note that in init we have only a LayoutInflater call and it’s per design. We will use spyk from Mockk to replace Android functionality with mock, while still maintaining our logic intact. The problem is that for spyk you need an already created instance which means that we cannot do anything except for the basic UI initialization in init. And also this is the reason why we move most of the actual setup code to initialize. It’s one of the drawbacks of the method: you’ll have to split logic and call initialization manually somewhere down the road.

Integrating view into Activity

This is a no-brainer, I add it here for the sake of completeness.

Layout:

<?xml version="1.0" encoding="utf-8"?>
<com.syllogismobile.unittestingview.CustomView
xmlns:android="http://schemas.android.com/apk/res/android&quot;
xmlns:tools="http://schemas.android.com/tools&quot;
android:id="@+id/customView"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity" />
view raw activity_main.xml hosted with ❤ by GitHub

Code:

import androidx.appcompat.app.AppCompatActivity
import android.os.Bundle
import kotlinx.android.synthetic.main.activity_main.*
class MainActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
customView.initialize()
updateCustomView()
}
private fun updateCustomView() {
val model = CustomViewModel(
"A cat's jaw has only up and down motion; it does not have any lateral, side to side motion, like dogs and humans.",
10
)
customView.update(model)
}
}
view raw MainActivity.kt hosted with ❤ by GitHub

As you can see, it’s just a wrapper to present our CustomView on the actual device with some mock data. 

Testing

Well, finally, we’ve arrived at the meat of the article. Let me present the testing code:

import android.content.Context
import android.view.LayoutInflater
import android.view.View
import android.widget.Button
import android.widget.LinearLayout.VERTICAL
import android.widget.TextView
import io.mockk.*
import io.mockk.impl.annotations.MockK
import kotlinx.android.synthetic.main.custom_view.view.*
import org.junit.AfterClass
import org.junit.Before
import org.junit.BeforeClass
import org.junit.Test
class CustomViewUnitTests {
@MockK private lateinit var context: Context
@MockK private lateinit var layoutInflater: LayoutInflater
@MockK private lateinit var fact: TextView
@MockK private lateinit var likes: TextView
@MockK private lateinit var likeButton: Button
private lateinit var subject: CustomView
@Before
fun setUp() {
MockKAnnotations.init(this, relaxed = true)
every { LayoutInflater.from(context) } returns layoutInflater
every { layoutInflater.inflate(any<Int>(), any()) } returns mockk()
subject = spyk(CustomView(context))
every { subject.context } returns context
every { subject.orientation = any() } returns Unit
every { subject.fact } returns fact
every { subject.likes } returns likes
every { subject.likeButton } returns likeButton
subject.initialize()
}
@Test
fun testInit() {
every { likes.text } returns "1"
val clickSlot = slot<View.OnClickListener>()
verify(exactly = 1) {
likeButton.setOnClickListener(capture(clickSlot))
subject.orientation = VERTICAL
}
clickSlot.captured.onClick(likeButton)
verify(exactly = 1) { likes.text = "2" }
}
@Test
fun testUpdatesCorrectly() {
val mockModel = CustomViewModel("text", 10)
subject.update(mockModel)
verify(exactly = 1) {
fact.text = "text"
likes.text = "10"
}
}
companion object {
@JvmStatic
@BeforeClass
fun setUpClass() {
mockkStatic(LayoutInflater::class)
}
@JvmStatic
@AfterClass
fun tearDownClass() {
unmockkStatic(LayoutInflater::class)
}
}
}
view raw CustomViewUnitTests.kt hosted with ❤ by GitHub

I think it’s better to start from the bottom, where the companion object is. As you can see, we define two functions: to run before the suite and to run after it. In first we make sure LayoutInflater is mocked (so we can mock LayoutInflater.from method later on) and in the second one, we unmock it to make sure other tests are not screwed. In the real scenario, you probably would have those methods in the base abstract class for all View tests.

Next, let’s look at the setUp method at the top of the suite. Here we initialize mocks declared above with @MockK annotation, create a subject for tests with spyk and then set up views for our subject with mocks. This is the second drawback of the method – since there is no “real” layout in tests we need to mock every single UI component we want to verify in tests. That can create quite a bit of boilerplate code, but at the same time allows you to fine-tune code to your needs. 

After this, tests themselves are quite trivial: in the first we verify logic in initialize method, in the second we verify the update method updates UI correctly. 

Closing words

I understand that this approach certainly won’t work for everyone, and it has its drawbacks. However, I do hope it will provide at least some developers with a better way to handle UI testing and maybe will spark a discussion that will result in a much better solution all together!

On this note, I would like to wish you all the best and, hopefully, I will see you in the next article!

Creating Custom LayoutInflater

Standard

Preface

I think LayoutInflater is one of the most under-researched and under-appreciated components of the Android framework. Which is ironic, in a way, since it’s used in pretty much every application ever written.

Anyway, in this short article I intend to show a simple example of custom LayoutInflater, discuss a few specifics of its inner workings, and hopefully, show why it’s worth using in your application!

Just give me the code

If you don’t really want to follow the article and would rather just poke the code by yourself here’s the repo with the sample to play with.

What is LayoutInflater

While LayoutInflater is used pretty much in every UI component on Android platform, more often than not it’s hidden under the hood (Activity’s getLayoutInflater) and when it’s used explicitly (RecyclerView’s Adapter) – it’s used as black-box.

So what does it do? Well, it’s quite straightforward – it translates XML layout into a View object. And also sometimes it swaps default implementation of views to their AppCompat counter-parts (not directly tho, i.e. there is no AppCompatLayoutInflater, see androidx/appcompat/app/AppCompatDelegateImpl.java instead). And also it uses reflection to create objects (and if you ever wondered why the creation of layout from XML is much slower than the creation of layout in code, here’s the part of the reason why). And also it has factories inside, so you won’t get away with just subclassing LayoutInflater.

Well in order to get it all together let’s concentrate on the practical example. In this article, we will focus on the creation of LayoutInflater which will allow us to define a custom attribute to parse and the behaviour based on the value of this attribute. In this specific example, we will make a “color” attribute which will apply colours from our custom theme, but obviously, you can come up with any other attributes and behaviours you want.

Defining Theme

This will be really straightforward, let’s define a data class with few colours inside:

import android.graphics.Color
data class CustomTheme(
val darkPurple: Int = Color.parseColor("#330C2F"),
val maximumPurple: Int = Color.parseColor("#7B287D"),
val violetBlueCrayola: Int = Color.parseColor("#7067CF"),
val lavenderBlue: Int = Color.parseColor("#B7C0EE"),
val aeroBlue: Int = Color.parseColor("#CBF3D2")
)
view raw CustomTheme.kt hosted with ❤ by GitHub

Defining layout

We will start with defining custom namespace like this:

xmlns:custom="https://syllogismobile.wordpress.com/"

Value is totally up to you, just make sure it starts with “http://” otherwise Android Studio will complain and throw a warning.

We will use attribute named “color” which value will be the name of colour defined in our custom theme above like this:

custom:color="darkPurple"

Our example layout will look like this:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android&quot;
xmlns:app="http://schemas.android.com/apk/res-auto&quot;
xmlns:tools="http://schemas.android.com/tools&quot;
xmlns:custom="https://syllogismobile.wordpress.com/&quot;
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="vertical"
tools:context=".MainActivity"
custom:color="darkPurple">
<TextView
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:text="Hello World!"
android:textSize="36sp"
android:gravity="center"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="parent"
custom:color="lavenderBlue"/>
<View
android:layout_width="match_parent"
android:layout_height="50dp"
custom:color="aeroBlue"/>
<TextView
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:text="Hello World!"
android:textSize="36sp"
android:gravity="center"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="parent"
custom:color="violetBlueCrayola"/>
</LinearLayout>
view raw activity_main.xml hosted with ❤ by GitHub

Note that custom namespace is not strictly required, you can use it like this:

color="violetBlueCrayola"

However, Android Studio will highlight it with two warnings (yes, two). As long as you don’t mind it, you can just don’t use a custom namespace. 

Implementing custom LayoutInflater

We will implement a very generic LayoutInflater which will accept a definition of attributes from the outside. Let’s start with the class definition.

Constructor

class CustomLayoutInflater(
context: Context,
private val parent: LayoutInflater = LayoutInflater.from(context)
) : LayoutInflater(parent, context)
view raw CustomLayoutInflater_Constructor.kt hosted with ❤ by GitHub

Note that by default we use the factory method from LayoutInflater, but we also allow to provide a custom value (for example, mock parent for tests).

Holding appliers

Next, we define a map that will hold appliers: lambdas that will apply the value of an attribute to the view:

private val registeredAppliers: MutableMap<String, (View, String) -> Unit> = mutableMapOf()

Injecting Factory2

Now we need to inject our implementation of Factory2 into inflater like this:

init {
factory2 = WrapperFactory(factory2)
}
view raw CustomLayoutInflater_init.kt hosted with ❤ by GitHub

WrapperFactory

WrapperFactory is defined like this:

inner class WrapperFactory(private val originalFactory: Factory2?) : Factory2 {
override fun onCreateView(name: String, context: Context, attrs: AttributeSet): View? {
return originalFactory?.onCreateView(name, context, attrs)?.apply { runAppliers(this, attrs) }
}
override fun onCreateView(parent: View?, name: String, context: Context, attrs: AttributeSet): View? {
return originalFactory?.onCreateView(parent, name, context, attrs)?.apply { runAppliers(this, attrs) }
}
}
view raw WrapperFactory.kt hosted with ❤ by GitHub

As you can see we simply use the original factory (which is set from the parent) and then run appliers if it returns an actual view.

Overriding onCreateView

Note that factory2 is taking precedence in view creation, however, it’s not going to be able to create the view all the time. For this case we need to override the method in the LayoutInflate itself like this:

override fun onCreateView(
name: String?,
attrs: AttributeSet?
): View? {
for (prefix in androidPrefixes) {
try {
val view = createView(name, prefix, attrs)
if (view != null) return view.apply { runAppliers(this, attrs) }
} catch (e: ClassNotFoundException) { }
}
return super.onCreateView(name, attrs)?.apply { runAppliers(this, attrs) }
}
view raw OnCreateView.kt hosted with ❤ by GitHub

Where androidPrefixes are:

private val androidPrefixes = listOf(
"android.widget.",
"android.webkit.",
"android.app."
)
view raw AndroidPrefixes.kt hosted with ❤ by GitHub

As you could’ve guessed, default Factory2 implementation won’t instantiate most usual Android views like LinearLayout, which is weird, but well, what can you do. 

Implementing runAppliers

Now let’s implement runAppliers:

private fun runAppliers(view: View, attrs: AttributeSet?) {
if (attrs == null) return
for (registeredTag in registeredAppliers.keys) {
attrs.getAttributeValue(NAMESPACE, registeredTag)?.let { value ->
registeredAppliers[registeredTag]?.let {
it(view, value)
}
}
}
}
view raw RunAppliers.kt hosted with ❤ by GitHub

Note that here we verify that attribute has a proper namespace. If you opted into approach without custom namespace – you can pass null instead of the namespace.

Putting it all together

Here’s whole code for CustomLayoutInflater:

import android.content.Context
import android.util.AttributeSet
import android.view.LayoutInflater
import android.view.View
class CustomLayoutInflater(
context: Context,
private val parent: LayoutInflater = LayoutInflater.from(context)
) : LayoutInflater(parent, context) {
private val registeredAppliers: MutableMap<String, (View, String) -> Unit> = mutableMapOf()
init {
factory2 = WrapperFactory(factory2)
}
fun registerApplier(tag: String, applier: (view: View, value: String) -> Unit): CustomLayoutInflater {
registeredAppliers[tag] = applier
return this
}
override fun cloneInContext(newContext: Context): LayoutInflater {
return CustomLayoutInflater(newContext, this)
}
override fun onCreateView(
name: String?,
attrs: AttributeSet?
): View? {
for (prefix in androidPrefixes) {
try {
val view = createView(name, prefix, attrs)
if (view != null) return view.apply { runAppliers(this, attrs) }
} catch (e: ClassNotFoundException) { }
}
return super.onCreateView(name, attrs)?.apply { runAppliers(this, attrs) }
}
private fun runAppliers(view: View, attrs: AttributeSet?) {
if (attrs == null) return
for (registeredTag in registeredAppliers.keys) {
attrs.getAttributeValue(NAMESPACE, registeredTag)?.let { value ->
registeredAppliers[registeredTag]?.let {
it(view, value)
}
}
}
}
inner class WrapperFactory(private val originalFactory: Factory2?) : Factory2 {
override fun onCreateView(name: String, context: Context, attrs: AttributeSet): View? {
return originalFactory?.onCreateView(name, context, attrs)?.apply { runAppliers(this, attrs) }
}
override fun onCreateView(parent: View?, name: String, context: Context, attrs: AttributeSet): View? {
return originalFactory?.onCreateView(parent, name, context, attrs)?.apply { runAppliers(this, attrs) }
}
}
companion object {
private val androidPrefixes = listOf(
"android.widget.",
"android.webkit.",
"android.app."
)
private const val NAMESPACE = "https://syllogismobile.wordpress.com/&quot;
fun from(context: Context): CustomLayoutInflater {
return CustomLayoutInflater(context)
}
}
}
view raw CustomLayoutInflater.kt hosted with ❤ by GitHub

Using inflater in Activity

This is very simple, so I’ll just show the code since it doesn’t have anything worth explaining:

import android.os.Bundle
import android.view.View
import android.widget.TextView
import androidx.appcompat.app.AppCompatActivity
class MainActivity : AppCompatActivity() {
private val theme = CustomTheme()
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(
CustomLayoutInflater
.from(this)
.registerApplier(COLOR_ATTRIBUTE, this::applyColorAttributeValue)
.inflate(R.layout.activity_main, null)
)
}
private fun applyColorAttributeValue(view: View, colorValue: String) {
val color = when(colorValue) {
"darkPurple" -> theme.darkPurple
"maximumPurple" -> theme.maximumPurple
"violetBlueCrayola" -> theme.violetBlueCrayola
"lavenderBlue" -> theme.lavenderBlue
"aeroBlue" -> theme.aeroBlue
else -> error("Unexpected color $colorValue")
}
when(view) {
is TextView -> view.setTextColor(color)
else -> view.setBackgroundColor(color)
}
}
private companion object {
const val COLOR_ATTRIBUTE = "color"
}
}
view raw MainActivity.kt hosted with ❤ by GitHub

As you can see we handle only TextView and View here, but you can extend/change it and add new attributes to handle as you wish.

Closing words

I hope this article managed to showcase the power of custom LayoutInflaters. While it is not the approach you would take in every single project, I think it’s worth considering for every project you’re working on!

Good luck and see you in the next articles (hopefully)!

Android, CMake and FFmpeg. Part Three: Throwing libx264 in the Mix

Standard

Series Contents

  1. Part One: CMake in Android Cookbook 
  2. Part Two: Building FFmpeg 
  3. Part Three: Throwing libx264 in the Mix [you are here]

Throwing libx264 in the Mix

Preface

This article was long coming. I actually intended to follow up my very original article on FFmpeg with a continuation on how to integrate libx264, but due to some circumstances, I wasn’t able to do it at that time. 

Anyway, the main purpose of the article is to show how to integrate external libraries into FFmpeg. While the main focus is on libx264, the general process should be extensible to any other library (hopefully). 

Getting started

Before we start let’s make sure we all on the same page here:

  • Make sure you have NDK installed. The version used in this article is 21.1.6352462.
  • Make sure you have CMake installed. The version used in this article is 3.10.2.
  • Make sure you have your C++ enabled project ready. 
  • Make sure you read the previous article in the series.

That’s pretty much all you need to get started.

Just give me the code

In case you’re not interested in the material in the article and would rather play with code, you can check out according to the article version of FFmpeg Development Kit.

Setting up

In this section, I will describe some groundwork before we will proceed to actual libx264 integration.

Adding switch to Gradle

Let’s add a switch to Gradle that will allow us to switch libx264 on and off. Remember that libx264 is distributed under GPL, so I think it’s a good idea to have an easy way to remove libx264 integration in case some legal problems arise.

Navigate your module’s build.gradle and add the following:

flavorDimensions "non-free"
productFlavors {
    free {
        dimension "non-free"
        buildConfigField "boolean", "LIB_X264_ENABLED", "false"
        externalNativeBuild {
            cmake {
                arguments "-DLIB_X264_ENABLED:BOOL=OFF", "-DANDROID_ARM_NEON=ON"
            }
        }
    }
    nonFree {
        dimension "non-free"
        buildConfigField "boolean", "LIB_X264_ENABLED", "true"
        externalNativeBuild {
            cmake {
                arguments '-DLIB_X264_ENABLED:BOOL=ON', "-DANDROID_ARM_NEON=ON"
            }
        }
    }
}

Nothing overly complicated here: we add a new flavor, add BuildConfig field that will indicate wherever libx264 is present or not for our Java/Kotlin code and we add a new flag for CMake which will indicate the same thing for the CMake configuration.

ffmpeg.cmake

We need a small adjustment to the ffmpeg.cmake. Navigate to the ExternalProject_Add call and at the very and of it, below 

LOG_INSTALL 1

Add:

DEPENDS ${FFMPEG_DEPENDS}

That will allow us to specify targets that must be built before FFmpeg is built (libx264 in our case).

CMakeLists.txt

Now navigate to the CMakeLists.txt and just before the line:

include(ffmpeg.cmake)

Add following code:

IF (${LIB_X264_ENABLED})
    include(libx264.cmake)

    set(FFMPEG_DEPENDS libx264_target)
    set(FFMPEG_EXTRA_C_FLAGS "-I${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/include")
    set(FFMPEG_EXTRA_LD_FLAGS "-L${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")

    list(APPEND FFMPEG_CONFIGURE_EXTRAS --enable-gpl --enable-libx264 --enable-nonfree)
    list(APPEND FFMPEG_LIBS postproc x264)
ENDIF()

Don’t worry about “libx264.cmake” for now, we will create it in the next step. For now, as you can see we set a new variable FFMPEG_DEPENDS which will allow us to indicate that we need to build something before FFmpeg is built. We also add an include path and the library path to the FFMPEG_EXTRA_C_FLAGS and FFMPEG_EXTRA_LD_FLAGS. 

The last but not least: we need to make sure FFmpeg is configured with proper flags and also add libraries to link against (postproc and x264).

libx264.cmake

Now, to the actual building. First of all, create a new file called “libx264.cmake”. By analogy with “ffmpeg.cmake” it’s going to be the file that will take care of building libx264 for us. Let’s start from the beginning (i.e. fetching).

Fetching sources

Add the following code to the newly created file:

cmake_minimum_required(VERSION 3.10.2)

set(LIBX264_URL https://code.videolan.org/videolan/x264/-/archive/master/x264-master.tar.bz2)

get_filename_component(LIBX264_ARCHIVE_NAME ${LIBX264_URL} NAME)
get_filename_component(LIBX264_NAME ${LIBX264_URL} NAME_WE)

IF (NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME})
    file(DOWNLOAD ${LIBX264_URL} ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_ARCHIVE_NAME})

    execute_process(
            COMMAND ${CMAKE_COMMAND} -E tar xzf ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_ARCHIVE_NAME}
            WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
    )

As you can see the process is very similar to the one we have for FFmpeg, with the exception that we don’t supply a version here and thus name handling works a little bit different.

Next, as with FFmpeg, we have to patch sources to make the library work properly on Android. You see by default libx264 creates a shared library with version suffix and libx264.so is a mere symlink. Android won’t load that properly (well it will, one time, but it seems to be removing symlinks after this). So let’s modify Makefile and “configure” script to get rid of that:

    file(READ ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}/configure configure_src)
    string(REPLACE "echo \"SONAME=libx264.so.$API\" >> config.mak" "echo \"SONAME=libx264.so\" >> config.mak" configure_src "${configure_src}")
    file(WRITE ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}/configure "${configure_src}")

    file(READ ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}/Makefile makefile_src)
    string(REPLACE "ln -f -s $(SONAME) $(DESTDIR)$(libdir)/libx264.$(SOSUFFIX)" "# ln -f -s $(SONAME) $(DESTDIR)$(libdir)/libx264.$(SOSUFFIX)" makefile_src "${makefile_src}")
    file(WRITE ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}/Makefile "${makefile_src}")
ENDIF()

What we do here is we modify configure to have SONAME without suffix and we modify Makefile to don’t create a symlink (i.e. we just comment it out in install step). That’s it for the fetching process.

Copying build system

The same way we had build system script for FFmpeg we will have it for libx264. Let’s create file called “libx264_build_system.cmake”. It will be almost the same as “ffmpeg_build_system.cmake” so I’ll just provide full code here:

cmake_minimum_required(VERSION 3.10.2)

if (${STEP} STREQUAL configure)
    # Encoding string to list
    string(REPLACE "|" ";" CONFIGURE_EXTRAS_ENCODED "${CONFIGURE_EXTRAS}")
    list(REMOVE_ITEM CONFIGURE_EXTRAS_ENCODED "")

    set(CONFIGURE_COMMAND
            ./configure
            --extra-cflags=${C_FLAGS}
            --extra-ldflags=${LD_FLAGS}
            --extra-asflags=${AS_FLAGS}
            --sysroot=${SYSROOT}
            --host=${HOST}
            --enable-pic
            --enable-shared
            --libdir=${LIBS_OUT}
            --libdir=${PREFIX}
            --prefix=${PREFIX}
            ${CONFIGURE_EXTRAS_ENCODED}
    )

    execute_process(COMMAND ${CONFIGURE_COMMAND})
elseif(${STEP} STREQUAL build)
    execute_process(COMMAND ${HOST_TOOLCHAIN}/make -j${NJOBS})
elseif(${STEP} STREQUAL install)
    execute_process(COMMAND ${HOST_TOOLCHAIN}/make install)
endif()

And in “libx264.cmake” add following code:

file(
        COPY ${CMAKE_CURRENT_SOURCE_DIR}/libx264_build_system.cmake
        DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}
        FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
)

Now we should have our build system script in the folder with sources.

Setting build tools

Similarly to the FFmpeg, we want to set our build tools for libx264:

set(LIBX264_CC ${CMAKE_C_COMPILER})
set(LIBX264_AR ${ANDROID_AR})
set(LIBX264_AS ${ANDROID_ASM_COMPILER})
set(LIBX264_RANLIB ${ANDROID_TOOLCHAIN_PREFIX}ranlib${ANDROID_TOOLCHAIN_SUFFIX})
set(LIBX264_STRIP ${ANDROID_TOOLCHAIN_ROOT}/bin/llvm-strip${ANDROID_TOOLCHAIN_SUFFIX})

Note that for libx264 we don’t need the NM tool. 

Setting flags

This section is the same as in “ffmpeg.cmake”:

string(REPLACE " -Wl,--fatal-warnings" "" LIBX264_LD_FLAGS ${CMAKE_SHARED_LINKER_FLAGS})

string(STRIP ${CMAKE_C_FLAGS} LIBX264_C_FLAGS)
string(STRIP ${LIBX264_LD_FLAGS} LIBX264_LD_FLAGS)

set(LIBX264_C_FLAGS "${LIBX264_C_FLAGS} --target=${ANDROID_LLVM_TRIPLE} --gcc-toolchain=${ANDROID_TOOLCHAIN_ROOT}")
set(LIBX264_ASM_FLAGS "${CMAKE_ASM_FLAGS} --target=${ANDROID_LLVM_TRIPLE}")
set(LIBX264_LD_FLAGS "${LIBX264_C_FLAGS} ${LIBX264_LD_FLAGS}")

Misc variables

Again, pretty much the same:

set(NJOBS 4)
set(HOST_BIN ${ANDROID_NDK}/prebuilt/${ANDROID_HOST_TAG}/bin)

Patching up x86

Well, this is where the fun begins I guess. Both FFmpeg and libx264 require patching on x86, but in a different manner. Where FFmpeg just requires you to disable assembly to comply with PIC, libx264 has PIC assembly on x86, but it uses too many registers (NDK passes -mstackrealign which takes up an additional register). And x86_64 requires NASM. YASM, unfortunately, is outdated and won’t work. So, what we will do is to check if the host has NASM installed and if not or ABI is x86 – we disable assembly:

set(LIBX264_CONFIGURE_EXTRAS "")
IF (${CMAKE_ANDROID_ARCH_ABI} MATCHES ^x86)
    find_program(NASM_EXE nasm)

    IF (NASM_EXE)
        SET(LIBX264_AS ${NASM_EXE})
        SET(LIBX264_ASM_FLAGS "")
    ENDIF()

    IF(NOT NASM_EXE OR ${CMAKE_ANDROID_ARCH_ABI} STREQUAL x86)
        list(APPEND LIBX264_CONFIGURE_EXTRAS --disable-asm)
    ENDIF()
ENDIF()

Note that we remove assembly flags for NASM – libx264 will set proper flags by itself.

Encoding extras

The same as in “ffmpeg.cmake”:

string(REPLACE ";" "|" LIBX264_CONFIGURE_EXTRAS_ENCODED "${LIBX264_CONFIGURE_EXTRAS}")

ExternalProject_Add

And now, finally, the main part of the script – defining build process itself:

ExternalProject_Add(libx264_target
        PREFIX libx264_pref
        URL ${CMAKE_CURRENT_SOURCE_DIR}/${LIBX264_NAME}
        DOWNLOAD_NO_EXTRACT 1
        CONFIGURE_COMMAND ${CMAKE_COMMAND} -E env
            CC=${LIBX264_CC}
            AS=${LIBX264_AS}
            AR=${LIBX264_AR}
            RANLIB=${LIBX264_RANLIB}
            STRIP=${LIBX264_STRIP}
            ${CMAKE_COMMAND}
                -DSTEP:STRING=configure
                -DHOST:STRING=${ANDROID_LLVM_TRIPLE}
                -DSYSROOT:STRING=${CMAKE_SYSROOT}
                -DC_FLAGS:STRING=${LIBX264_C_FLAGS}
                -DAS_FLAGS:STRING=${LIBX264_ASM_FLAGS}
                -DLD_FLAGS:STRING=${LIBX264_LD_FLAGS}
                -DPREFIX:STRING=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
                -DCONFIGURE_EXTRAS:STRING=${LIBX264_CONFIGURE_EXTRAS_ENCODED}
            -P libx264_build_system.cmake
        BUILD_COMMAND ${CMAKE_COMMAND}
            -DSTEP:STRING=build
            -DNJOBS:STRING=${NJOBS}
            -DHOST_TOOLCHAIN:STRING=${HOST_BIN}
        -P libx264_build_system.cmake
        BUILD_IN_SOURCE 1
        INSTALL_COMMAND ${CMAKE_COMMAND}
            -DSTEP:STRING=install
            -DHOST_TOOLCHAIN:STRING=${HOST_BIN}
        -P libx264_build_system.cmake
        LOG_BUILD 1
        LOG_INSTALL 1
        LOG_CONFIGURE 1
)

It is in general very similar to the “ffmpeg.cmake” version, however, note that here we export all build tools as environment variables for the configuration step. We do it because libx264 doesn’t allow us to provide it as “configure” params. The only parameter that kind of allows us to do that is “cross-prefix”, but it won’t work with NASM on x86_64 then.

Closing words

This series of articles was long coming and I’m glad I finally was able to finish it, even replacing ndk-build with CMake. 

I do hope it will help out other programmers, smashing their head against the wall of native development, as I was, to finish their work faster and easier!

See you in the next articles!

Android, CMake and FFmpeg. Part Two: Building FFmpeg

Standard

Series Contents

  1. Part One: CMake in Android Cookbook 
  2. Part Two: Building FFmpeg [you are here]
  3. Part Three: Throwing libx264 in the Mix

Building FFmpeg

Preface

This article expands on my previous works and improves them in a few areas (specifically performance-wise, because the previous article used –disable-asm). CMake workflow makes FFmpeg integration much smoother and easier to handle since you’re working in one place, instead of constantly jumping between terminal windows as with ndk-build. 

While I hope the material in this article is self-sufficient, so you won’t need to read previous articles on FFmpeg, I think it still might be worth it to glance them over just to refresh some basic vocabulary.

Getting started

Before we start let’s make sure we all on the same page here:

  • Make sure you have NDK installed. The version used in this article is 21.1.6352462.
  • Make sure you have CMake installed. The version used in this article is 3.10.2.
  • Make sure you have your C++ enabled project ready. 

That’s pretty much all you need to get started.

Just give me code

In case you’re not interested in the material in the article and would rather play with code, you can check out according to the article version of FFmpeg Development Kit.

Setting up for the build process

Navigate to your CMakeLists.txt file and add following code here:

link_directories(${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
include_directories(${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/include)

We’re going to build libraries into ${CMAKE_LIBRARY_OUTPUT_DIRECTORY} and headers into ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/include. Here we make sure that all targets will be able to find our libraries and headers later on.

Next, make sure you have included ExternalProject like so:

include(ExternalProject)

Now, let’s define all the libraries we will use:

set(FFMPEG_LIBS avutil swresample avcodec avformat swscale avfilter avdevice)

We define it as a separate variable both for readability sake and ability to change it dynamically later on, if necessary. 

Also, let’s add an additional QoL variable which will allow us to pass additional configure flags from the main script to the configure process like this:

set(FFMPEG_CONFIGURE_EXTRAS )

You can add a list of additional flags here as a list.

This is pretty much all for the setting up process in the main script. Let’s move all the work related to the actual FFmpeg building to a separate script to make it cleaner and easier to maintain.

ffmpeg.cmake

Preparing the sources

Create a file in the same folder your CMakeLists.txt resides and call it “ffmpeg.cmake”. This is going to be the module that will build FFmpeg for us.

Let’s start from the start i.e. actually getting sources to build. First, we will define a few support variables to make our life easier:

set(FFMPEG_VERSION 4.2.2)
set(FFMPEG_NAME ffmpeg-${FFMPEG_VERSION})
set(FFMPEG_URL https://ffmpeg.org/releases/${FFMPEG_NAME}.tar.bz2)

You can change FFMPEG_VERSION to whatever version you want, but be aware that it might not work.

Next, let’s extract archive name from the URL:

get_filename_component(FFMPEG_ARCHIVE_NAME ${FFMPEG_URL} NAME)

You might wonder why won’t we simply set it like that:

set(FFMPEG_ARCHIVE_NAME ffmpeg-${FFMPEG_VERSION}.tar.bz2)
set(FFMPEG_URL https://ffmpeg.org/releases/${FFMPEG_ARCHIVE_NAME})

My reasoning is that I want to allow URL to change without breaking too much logic, however you’re free to take any approach you want.

Now let’s do actual fetching. First we check if we already have downloaded the sources:

IF (NOT EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME})

Note that we use CMAKE_CURRENT_SOURCE_DIR to make sure we have an absolute path to the folder. The reason is that your scripts are re-configured multiple times in the “cxx” folder for different flavors and architectures. To avoid excess work and messing up it’s better to always use absolute paths, because relative paths will be evaluated relative to cache directory (inside “cxx”).

Now we want to download archive:

file(DOWNLOAD ${FFMPEG_URL} ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_ARCHIVE_NAME})

But we can’t use the archive – we need sources inside it. Let’s unpack it:

execute_process(
            COMMAND ${CMAKE_COMMAND} -E tar xzf ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_ARCHIVE_NAME}
            WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)

Note that we use ${CMAKE_COMMAND} purely for cross-platform reasons. If you prefer another unpacking method and your host will always be the same – you can change COMMAND to any other extraction command you want.

Well, so far so good, but not entirely. We need to patch sources a little bit to make them usable on Android. You see, “main” inside ffmpeg.c has “exit” invocation just before return. In shell programs, it’s totally fine (as well as calling abort). The problem is that on Android it’s treated as a call to close the entire app. Which means your app will just silently close on the user as soon as processing ends. Talking about great UX huh. 

So let’s remove that:

file(READ ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg.c ffmpeg_src)

string(REPLACE "exit_program(received_nb_signals ? 255 : main_return_code);" "//exit_program(received_nb_signals ? 255 : main_return_code);" ffmpeg_src "${ffmpeg_src}")
string(REPLACE "return main_return_code;" "return received_nb_signals ? 255 : main_return_code;" ffmpeg_src "${ffmpeg_src}")

file(WRITE ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg.c "${ffmpeg_src}")

As you can see we simply comment out the exit call and move proper return code to the return. 

Well after that we finished with the fetching process, let’s close IF with ENDIF().

Now let’s add the build system script to the fetched folder. Note that we won’t create it just yet, it will just be a CMake wrapper around configuration and build process. 

file(
        COPY ${CMAKE_CURRENT_SOURCE_DIR}/ffmpeg_build_system.cmake
        DESTINATION ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}
        FILE_PERMISSIONS OWNER_READ OWNER_WRITE OWNER_EXECUTE GROUP_READ GROUP_EXECUTE WORLD_READ WORLD_EXECUTE
)

Don’t worry about the script, for now, we’re going to add it later on.

Configuring build tools

This section is rather simple – we just want to configure specific build tools (C compiler, ASM compiler etc.) for the FFmpeg. Let’s start with tools that already defined in the toolchain:

set(FFMPEG_CC ${CMAKE_C_COMPILER})
set(FFMPEG_CXX ${CMAKE_CXX_COMPILER})
set(FFMPEG_AR ${ANDROID_AR})
set(FFMPEG_AS ${ANDROID_ASM_COMPILER})

We essentially make copies of those variables so if we ever would want to change them (for example, setting different C compiler for specific ABI) – we could do that without breaking stuff in other places of the build process (remember that variables exposed by toolchain are global and it’s probably not a good idea to modify them).

Now let’s move to stuff that toolchain doesn’t expose yet:

set(FFMPEG_RANLIB ${ANDROID_TOOLCHAIN_PREFIX}ranlib${ANDROID_TOOLCHAIN_SUFFIX})
set(FFMPEG_STRIP ${ANDROID_TOOLCHAIN_ROOT}/bin/llvm-strip${ANDROID_TOOLCHAIN_SUFFIX})
set(FFMPEG_NM ${ANDROID_TOOLCHAIN_PREFIX}nm${ANDROID_TOOLCHAIN_SUFFIX})

Note that this set of build tools is specific to FFmpeg. Your particular case might not need ranlib or nm, for instance, so make sure to check beforehand what you need to provide to don’t have useless declarations.

Setting the flags

The first thing we want to do is remove “-Wl,–fatal-warnings” from linker flags provided by the toolchain. While this flag might be ok for your library it most definitely prevents FFmpeg from building. We can do it like this:

string(REPLACE " -Wl,--fatal-warnings" "" FFMPEG_LD_FLAGS ${CMAKE_SHARED_LINKER_FLAGS})

Nothing fancy here. Since toolchain doesn’t provide a way to turn it off – we just replace this part of the string with an empty string.

Now let’s proceed to setting flags for the FFmpeg:

set(FFMPEG_C_FLAGS "${CMAKE_C_FLAGS} --target=${ANDROID_LLVM_TRIPLE} --gcc-toolchain=${ANDROID_TOOLCHAIN_ROOT} ${FFMPEG_EXTRA_C_FLAGS}")
set(FFMPEG_ASM_FLAGS "${CMAKE_ASM_FLAGS} --target=${ANDROID_LLVM_TRIPLE} ${FFMPEG_EXTRA_ASM_FLAGS}")
set(FFMPEG_LD_FLAGS "${FFMPEG_C_FLAGS} ${FFMPEG_LD_FLAGS} ${FFMPEG_EXTRA_LD_FLAGS}")

Again, we duplicate flags so we avoid modifying flags set by toolchain and accidentally messing up other targets. Note that we have additional variables for every flag to provide additional flags in case we need to (FFMPEG_EXTRA_*_FLAGS). Also, note that LD flags also include C flags. I figured it out from the build.ninja, it seems to be necessary. 

Setting additional variables

Let’s define a few additional variables that will help us control the build process:

set(NJOBS 4)
set(HOST_BIN ${ANDROID_NDK}/prebuilt/${ANDROID_HOST_TAG}/bin)

The NJOBS variable will control how many jobs Make is going to use (or how much of your CPU capacity it’s going to use, in other terms).

HOST_BIN has a path to the host binaries inside the NDK. The reason we need this path is that we want to use Make provided by NDK, not the actual host one (assuming the host even has it installed, which is not guaranteed). 

Patching up x86

FFmpeg has text relocations by design on x86, while Android from Marshmallow and up explicitly forbids having text relocations in native libraries. Since x86 is a really limited platform (you probably only would see it in the emulator and I recommend using x86_64 images instead of x86 anyway) so we just disable assembly on the x86 ABI.

IF (${CMAKE_ANDROID_ARCH_ABI} STREQUAL x86)
    list(APPEND FFMPEG_CONFIGURE_EXTRAS --disable-asm)
ENDIF()

Encoding extras

We will have to pass extras to the script, however, there is a problem here. By default CMake uses “;” as a separator for both lists (which are strings under the hood) and for separating arguments for the CMake invocations. Which means if we just pass a list as the argument for the CMake script we will end up with all flags being treated as separate arguments.

So in order to avoid that we will just encode list with different separator:

string(REPLACE ";" "|" FFMPEG_CONFIGURE_EXTRAS_ENCODED "${FFMPEG_CONFIGURE_EXTRAS}")

Note that I used “|” as a separator since it’s unlikely to appear in the flags, however, if it conflicts with your use case – feel free to change it to any other symbol.

Configuring ExternalProject

Now we have everything we need to actually configure FFmpeg for building. There will be a lot of code, however much of it is explained in part one of this series. I’ll concentrate only on specific to the FFmpeg parts:

ExternalProject_Add(ffmpeg_target
        PREFIX ffmpeg_pref
        URL ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}
        DOWNLOAD_NO_EXTRACT 1
        CONFIGURE_COMMAND ${CMAKE_COMMAND} -E env
            PATH=${ANDROID_TOOLCHAIN_ROOT}/bin:$ENV{PATH}
            AS_FLAGS=${FFMPEG_ASM_FLAGS}
            ${CMAKE_COMMAND}
            -DSTEP:STRING=configure
            -DARCH:STRING=${CMAKE_SYSTEM_PROCESSOR}
            -DCC:STRING=${FFMPEG_CC}
            -DSTRIP:STRING=${FFMPEG_STRIP}
            -DAR:STRING=${FFMPEG_AR}
            -DAS:STRING=${FFMPEG_AS}
            -DNM:STRING=${FFMPEG_NM}
            -DRANLIB:STRING=${FFMPEG_RANLIB}
            -DSYSROOT:STRING=${CMAKE_SYSROOT}
            -DC_FLAGS:STRING=${FFMPEG_C_FLAGS}
            -DLD_FLAGS:STRING=${FFMPEG_LD_FLAGS}
            -DPREFIX:STRING=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
            -DCONFIGURE_EXTRAS:STRING=${FFMPEG_CONFIGURE_EXTRAS_ENCODED}
        -P ffmpeg_build_system.cmake
        BUILD_COMMAND ${CMAKE_COMMAND} -E env
            PATH=${ANDROID_TOOLCHAIN_ROOT}/bin:$ENV{PATH}
            ${CMAKE_COMMAND}
            -DSTEP:STRING=build
            -NJOBS:STRING=${NJOBS}
            -DHOST_TOOLCHAIN:STRING=${HOST_BIN}
        -P ffmpeg_build_system.cmake
        BUILD_IN_SOURCE 1
        INSTALL_COMMAND ${CMAKE_COMMAND} -E env
            PATH=${ANDROID_TOOLCHAIN_ROOT}/bin:$ENV{PATH}
            ${CMAKE_COMMAND}
            -DSTEP:STRING=install
            -DHOST_TOOLCHAIN:STRING=${HOST_BIN}
        -P ffmpeg_build_system.cmake
        STEP_TARGETS copy_headers
        LOG_CONFIGURE 1
        LOG_BUILD 1
        LOG_INSTALL 1
)

In CONFIGURE_COMMAND we invoke ffmpeg_build_system.cmake with STEP “configure” and all parameters required for the configuration process. LIBS_OUT and HEADERS_OUT parameters allow us to specify where libs and headers will be installed and TARGET_TOOLCHAIN is needed for “yasm” to be found (we add it to PATH).

In BUILD_COMMAND we supply STEP equal to “build” to indicate what we are doing,  NJOBS to control how much resources Make can use and HOST_TOOLCHAIN to have NDK’s Make available. TARGET_TOOLCHAIN is needed to make sure we have yasm available if needed.

In INSTALL_COMMAND we supply STEP equal to “install” to indicate what we are doing, TARGET_TOOLCHAIN and HOST_TOOLCHAIN are for the same reasons as in the build step.

In STEP_TARGETS we define an additional step that will take place before the “install” step and it will copy all headers from the FFmpeg’s folder to the headers installation folder. The reason is that since we want to use CLI tools, which use some internal stuff, we have to copy headers in order to make them work.

LOG_* properties just set to make sure if something fails we can diagnose what happened.

Adding step for headers’ copying

As mentioned above we need this step to make CLI tools work inside our library. Let’s start with creating a script that will do the actual copying. 

copy_headers.cmake

In the same folder where your CMakeLists.txt resides create a file called “copy_headers.cmake”. This script will be very simple, just a few commands:

cmake_minimum_required(VERSION 3.10.2)  

file(GLOB libs "${SOURCE_DIR}/${FFMPEG_NAME}/lib*")
file(
        COPY ${libs} ${BUILD_DIR}/config.h ${SOURCE_DIR}/${FFMPEG_NAME}/compat
        DESTINATION ${OUT}/include
        FILES_MATCHING PATTERN *.h
)

Few notes. Firstly, we use ${SOURCE_DIR} which is passed as a parameter, not ${CMAKE_CURRENT_SOURCE_DIR}. The reason is that if we use CMAKE_CURRENT_SOURCE_DIR it will point to the cache directory since the script will be executed at build time, not at configure time. Secondly, we use GLOB to get a list of all libraries folders in the FFmpeg’s sources folder and then expand this list in the copy invocation. 

We provide two additional variables: BUILD_DIR, which points to the FFmpeg’s build folder with proper “config.h”, and OUT which points to the directory where libs will be installed (essentially, it is CMAKE_LIBRARY_OUTPUT_DIRECTORY).

Also, it’s worth noting we copy only headers (*.h files), since we don’t need source files there.

Executing script as custom step

Now let’s add actual custom step back in the “ffmpeg.cmake” file to run the script:

ExternalProject_Get_property(ffmpeg_target SOURCE_DIR)
ExternalProject_Add_Step(
        ffmpeg_target
        copy_headers
        COMMAND ${CMAKE_COMMAND}
            -DBUILD_DIR:STRING=${SOURCE_DIR}
            -DSOURCE_DIR:STRING=${CMAKE_CURRENT_SOURCE_DIR}
            -DFFMPEG_NAME:STRING=${FFMPEG_NAME}
            -DOUT:STRING=${CMAKE_LIBRARY_OUTPUT_DIRECTORY}
        -P ${CMAKE_CURRENT_SOURCE_DIR}/copy_headers.cmake
        DEPENDEES build
        DEPENDERS install
)

First of all, we’re getting the build directory with ExternalProject_Get_property and putting it in the property called SOURCE_DIR (note that since we build in source and build directories are the same). Then we add the actual implementation (definition is in the ExternalProject_Add) of the step with script invocation. The last piece of the puzzle is that we set it to run between “build” and “install” steps explicitly by defining DEPENDEES and DEPENDERS properties. 

This finalizes the build process of FFmpeg itself, leaving us to do the integration part.

Defining sources for the CLI tools

We define the list of sources to use later on in the main script:

set(ffmpeg_src
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/cmdutils.c
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg_cuvid.c
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg_filter.c
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg_hw.c
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg_opt.c
        ${CMAKE_CURRENT_SOURCE_DIR}/${FFMPEG_NAME}/fftools/ffmpeg.c
)

Nothing really to explain here, we just define a path to sources for later use.

ffmpeg_build_system.cmake

There are a few reasons I decided to use a separate script. The first reason is that I had an insane amount of trouble with getting just plain “./configure” to run. Shell wasn’t able to run the script, throwing out “command not found”, parameters weren’t supplied correctly etc. Using CMake script seems to be a better and more stable option. The second reason is that there is some additional logic we might want to have in the script, like exporting variables, modifying flags and so on.

The script is actually very simple:

cmake_minimum_required(VERSION 3.10.2)

if (${STEP} STREQUAL configure)
    # Encoding string to list
    string(REPLACE "|" ";" CONFIGURE_EXTRAS_ENCODED "${CONFIGURE_EXTRAS}")
    list(REMOVE_ITEM CONFIGURE_EXTRAS_ENCODED "")

    # Note that we don't pass LD, Clang sets it internally based of --target
    set(CONFIGURE_COMMAND
            ./configure
            --cc=${CC}
            --ar=${AR}
            --strip=${STRIP}
            --ranlib=${RANLIB}
            --as=${AS}
            --nm=${NM}
            --target-os=android
            --arch=${ARCH}
            --extra-cflags=${C_FLAGS}
            --extra-ldflags=${LD_FLAGS}
            --sysroot=${SYSROOT}
            --enable-cross-compile
            --disable-static
            --disable-programs
            --disable-doc
            --enable-shared
            --enable-protocol=file
            --enable-pic
            --shlibdir=${PREFIX}
            --prefix=${PREFIX}
            ${CONFIGURE_EXTRAS_ENCODED}
    )

    execute_process(COMMAND ${CONFIGURE_COMMAND})
elseif(${STEP} STREQUAL build)
    execute_process(COMMAND ${HOST_TOOLCHAIN}/make -j${NJOBS})
elseif(${STEP} STREQUAL install)
    execute_process(COMMAND ${HOST_TOOLCHAIN}/make install)
endif()

Build and install steps just run “make” and “make install”. Configure step is a little bit more complicated, but also not extremely hard. Essentially here we decode extras back to the list, then we append those to “configure” invocation. 

Back to the CMakeLists.txt

Now we actually can go to the main script and link everything we’ve done so far in “ffmpeg.cmake” to our actual library.

First of all, include “ffmpeg.cmake”:

include(ffmpeg.cmake)

Then modify your add_library call to include ffmpeg_src, like so:

add_library(native-lib SHARED native-lib.c ${ffmpeg_src})

Next, we need to make sure FFmpeg actually is going to be built. By default, CMake in Android will build only the main target. In order to make sure FFmpeg is built we need to specify FFmpeg’s target as a dependency to your main target like so:

add_dependencies(native-lib ffmpeg_target)

And the last, but not least, we have to actually link all built libraries against our main library:

target_link_libraries(native-lib ${FFMPEG_LIBS})

That will allow us to use libraries in our library.

Using FFmpeg in the library

This is pretty much the same as it was in previous articles. Let’s define a function that will call FFmpeg’s main function:

int main(int argc, char **argv);

jint run_ffmpeg(
        JNIEnv *env,
        jobjectArray args
) {
    int i = 0;
    int argc = 0;
    char **argv = NULL;
    jstring *strr = NULL;

    if (args != NULL) {
        argc = (*env)->GetArrayLength(env, args);
        argv = (char **) malloc(sizeof(char *) * argc);
        strr = (jstring *) malloc(sizeof(jstring) * argc);

        for (i = 0; i < argc; ++i) {
            strr[i] = (jstring)(*env)->GetObjectArrayElement(env, args, i);
            argv[i] = (char *)(*env)->GetStringUTFChars(env, strr[i], 0);
        }
    }

    jint retcode = 0;
    retcode = main(argc, argv);

    for (i = 0; i < argc; ++i) {
        (*env)->ReleaseStringUTFChars(env, strr[i], argv[i]);
    }

    free(argv);
    free(strr);

    return retcode;
}

Note that we have a forward declaration of “main” function before ours to make code compile properly. You can call the “run_ffmpeg” function inside your JNI function, just make sure to pass the correct list of arguments. 

Don’t forget to load libraries at runtime before you call your JNI function like this:

System.loadLibrary("avutil")
System.loadLibrary("swresample")
System.loadLibrary("avcodec")
System.loadLibrary("avformat")
System.loadLibrary("swscale")
System.loadLibrary("avfilter")
System.loadLibrary("avdevice")

That’s pretty much it, now you should be able to build and run FFmpeg (hopefully).

Bonus content

This section describes a few additional things that are not exactly needed in the main article, but still are super useful.

Getting logs from native libs

The credit for this goes to this SO answer. 

Essentially the problem is that somebody in the early days of Android decided that nobody would ever need stdout and stderr streams and redirected them to /dev/null. The recommended approach is to use Android’s logging library, which is not feasible in case of big third-party libraries (and pretty much impossible if the library is not open-sourced). So what can we do?

The only working option I was able to find is to redirect streams to the Logcat like this:

#include <android/log.h>

static int pfd[2];
static pthread_t thr;

static void* thread_func(void* in) {
    ssize_t rdsz;
    char buf[128];
    while((rdsz = read(pfd[0], buf, sizeof buf - 1)) > 0) {
        if(buf[rdsz - 1] == '\n') --rdsz;
        buf[rdsz] = 0;  /* add null-terminator */
        __android_log_write(ANDROID_LOG_DEBUG, tag, buf);
    }
    return 0;
}

jint JNI_OnLoad(JavaVM* vm, void* reserved){
    setvbuf(stdout, 0, _IOLBF, 0);
    setvbuf(stderr, 0, _IONBF, 0);

    pipe(pfd);
    dup2(pfd[1], 1);
    dup2(pfd[1], 2);

    if(pthread_create(&thr, 0, thread_func, 0) == -1) return JNI_VERSION_1_6;

    pthread_detach(thr);

    return JNI_VERSION_1_6;
}

First, we assign buffers to both stdout and stderr with setvbuf calls. Then we create a pipe and assign both stdout and stderr to the write end of the pipe. Then we start a thread to pool read end of the pipe and write everything it gets to the Logcat.

That’s pretty much it, now you should see logs from native lib in your Logcat. Don’t forget to link the log library to your native library for this to work!

Enabling MediaCodec support in FFmpeg

FFmpeg has supported MediaCodec for a while now, however, due to the nature of this particular library, it might not be suited for every format of the video. Usually, MediaCodec works well only with videos recorded on the device, so I would suggest either re-encoding video received from the internet to make sure it matches expected profile (like h264) or don’t use it at all since it might cause your app to receive native crash (because FFmpeg might call abort).

Anyway, if you want to try it out here’s what you have to do. Make sure FFmpeg is configured with the following flags “target-os=android”, “–enable-jni” and “–enable-mediacodec”. Then add the following call to your native lib JNI_OnLoad:

 av_jni_set_java_vm(vm, NULL);

And make sure to include:

#include <libavcodec/jni.h>

That’s pretty much it, you now should be able to select MediaCoded as the codec for encoding/decoding in the FFmpeg call. But as I said be careful, it might crash the app!

Android, CMake and FFmpeg. Part One: CMake in Android Cookbook

Standard

Series Preface

This series of three articles (series content below) aims to help Android developers without much of experience in the native development to integrate external native libraries (in this case FFmpeg and libx264, but it should be relatively easy to extend to other projects) and streamline the whole process by using CMake instead of previously used ndk-build.

Series Contents

  1. Part One: CMake in Android Cookbook [you are here]
  2. Part Two: Building FFmpeg
  3. Part Three: Throwing libx264 in the Mix

CMake in Android Cookbook

Preface

It’s important to highlight that it is not really an article or a tutorial on either CMake nor its specifics on the Android platform. As the name, “Cookbook”, suggests it’s supposed to be used as a reference to be returned to multiple times and it’s mixed both from “beginners” tips and more advanced stuff (so if it seems that a section talks about something you already know, feel free to skip it). 

It’s also worth noting that while it’s part of a series, mainly because it is a necessary foundation to make sense of what is happening in next parts, I do hope it will be useful to everybody, who is working or plans to work in CMake!

CMake: getting started

I think the best way to describe CMake is “more elaborate version of configure-based approach”. Essentially, it configures projects for specific build systems (Make, Ninja, Visual Studio, XCode etc.) using a specially designed scripting language (note that Android Studio uses Ninja, the build system designed for fast, iterative builds).

How a project is generated is defined by a file called CMakeLists.txt, which you can think of as a project file (or script, rather) of sorts. If you used the Native C++ template in the New Project Wizard in Android Studio it should’ve already generated the bare bones CMake project for you. 

If you have an existing project you wish to integrate native code to, you can follow this guide to navigate you through.

The next section will present basic syntax and some commands that will be used later on.

CMake: syntax and commands reference

Variables

CMake has a relatively complex system for variables. I suggest referring to set documentation for additional details. 

Let’s start with the basics.

Basics

set(<name>, <value>) – sets value to the variable, where <value> can be string, number or list.

To set a list simply call set(<name>, <value_0> <value_1> … <value_n>).  Note that lists are handled as strings by CMake internally, with elements separated by “;”. So calls set(list, a b c) and set(list, “a;b;c”) are the same. 

To get a variable you just need to put its name in an expansion operator like this: ${<name>}. 

Important note: people who worked with shell scripts before might tend to use “${<name>}” to make sure value is expanded correctly without really thinking if they need a string here or not. I strongly advise from doing this: CMake has a lot of internal magic, which splits arguments for commands and saves them into the cache and there is a good chance you’re going to break it by making it treat some parts of command as unsplittable values. A good rule of thumb would be: don’t use quotes unless it doesn’t work without them. While you’re probably not going to run into issues in simple cases, in more complex scenarios (for example making custom targets) it might create all kinds of issues and cause hours of painstaking debugging to fix the issue. 

Setting and getting environment variables

To set environment variables in code use: set(ENV{<name>} <value>).

To get environment variable use: $ENV{<name>}.

Setting variables in script invocation

Often you would need to invoke CMake script with certain flags or values. You can do so with -D switch. For example:

${CMAKE_COMMAND} -DNAME:STRING=value -P script.cmake

Will invoke script.cmake with variable NAME set to “value”. 

To set environment variables for the script use “-E env” (I suggest looking at Command-Line Tool Mode for CMake for more details) switch which allows you to modify the environment for the invocation:

 ${CMAKE_COMMAND} -E env PATH=some_path:$ENV{PATH} <COMMAND>

Controlling execution flow

Controlling program flow (if, for and while) is more in line with what you would expect from a programming language, but there are also a few differences to the syntax you would usually see.

Note that in order to break loops you can use break() and continue().

IF

Controlling flow with if:

IF(<condition>)
    <code>
ELSEIF(<condition>)
    <code>
ENDIF()

Possible conditions are:

  • <variable>|<value> LESS|GREATER|EQUAL <variable>|<value> for numbers
  • <variable>|<value> STREQUAL|MATCHES <variable>|<value> for strings
  • EXISTS <path> check if path exists
  • NOT <condition> invert
  • <condition> AND|OR <condition> logic operators

This is most definitely not a complete list of operators, but most useful ones, in my opinion. Note that <variable> here is variable name, not expansion (i.e. name, not ${name}).

FOREACH

To iterate over list use following syntax:

FOREACH(<variable> IN <list>)
    <loop>
ENDFOREACH(<variable>)

To iterate with index you have following options:

FOREACH(<variable> RANGE <total>)
    <loop>
ENDFOREACH(<variable>)

or with specific start, stop and optional step:

FOREACH(<variable> RANGE <start> <stop> [step])
    <loop>
ENDFOREACH(<variable>)

WHILE

This is pretty straightforward, you can use it in following way:

WHILE(<condition>)
    <code>
ENDWHILE()

Useful commands

I think it’s important to note that everything described above (including set) are also commands. But I think it’s easier mentally to distinguish syntax constructions and other commands, especially when you’re just starting out and trying to apply knowledge from other languages to CMake.

file()

File is a very useful command that allows you to manipulate local and remote file systems. I’ll give general outline of available subcommands that I find the most useful:

  • file(READ <path> <variable>) – reads file at path to the variable.
  • file(WRITE|APPEND <path> <content>) – writes/appends content to file.
  • file(GLOB <variable> <expr>) – collects files matching <expr> into <variable> in specified folder.
  • file(GLOB_RECURSE <variable> <expr>) – collects files matching <expr> into <variable> in specified folder and all subfolders.
  • file(TO_CMAKE_PATH <path> <variable>) – converts path to internal CMake format. Helpful on Windows which has backslashes in the path.
  • file(DOWNLOAD <url> <path>) – download file from <url> to <path>.

In order to find files under a certain folder with GLOB you can use expressions like “path/expression”, i.e. just prefix your expression with path, otherwise it will run in the current directory. However, I should note that if you want to do this to add sources to the target – it’s not advised to do so by CMake creators. The main reason is that CMake won’t pick up if you have changed the target sources and will not prompt you to re-run configuration. If you work with co-workers, after they download changes it won’t prompt them either to re-run configuration, so it might create some weird issues with the project not building properly. It doesn’t necessarily mean you absolutely should not use it ever, just be aware of that possibility and make a conscious decision. 

find_library and find_program

Well those two are pretty straightforward: they find a library (to link against) or program (to use).

You can run them with: find_(library|program)(<variable> <name>) and it will run search through PATH and CMake search locations and if the library/program is found its path will be put into <variable>.

get_filename_component

A super useful command to deal with paths/urls. Basic usage is:

get_filename_component(<variable> <path/url> <mode>)

It will put part of the path/url specified by mode into <variable>.

Mode is one of the following:

  • DIRECTORY – path to file directory.
  • NAME – file name (with extension).
  • EXT – file extension.
  • NAME_WE – file name (without extension).
  • LAST_EXT  – last file extension (i.e. c from a.b.c).
  • NAME_WLE  – file name (with last extension).

include

Includes CMake file or module into the project. For example: include(your_file.cmake) or include(CMakeModule) (a little bit more about built-in modules later on).

Important note: it preserves the scope of the parent project, which effectively means all variables and functions declared before include() invocation are also available in the included script.

math

Allows you to evaluate mathematical expression. Example usage:

math(EXPR <variable> “100 * 2”)

will put 200 into <variable>.

Note that the result must be representable as a 64-bit signed integer.

list

List is a helper command to work with lists which has a lot of useful sub-commands.

  • list(LENGTH <list> <variable>) – puts length of <list> into <variable>.
  • list(GET <list> <index> <variable>) – puts element at <index> from <list> to <variable>. You can specify multiple indexes to get a subset of the original list.
  • list(APPEND <list> <element>) – appends an <element> to <list>. You can specify multiple elements to append them to list.
  • list(REMOVE_AT <list> <index>) – removes element at <index> from <list>. You can pass multiple indexes to remove multiple items.

string

Very useful command to operate on strings. 

  • string(REPLACE <match_string> <replace_string> <variable> <input>) – finds <match_string> in <input>, replaces it with <replace_string> and puts result into <variable>.
  • string(REGEX REPLACE <match_expr> <replace_expr> <variable> <input>) – finds expression <match_expr> in <input>, replaces it with <replace_expr> and places result in the <variable>.
  • string(APPEND <value> <input>) – appends string to the string.
  • string(STRIP <string> <variable>) – strips whitespaces from <string> and puts result into <variable>.

add_dependecies

Allows you to specify dependencies between targets. Basic usage looks like: 

add_dependecies(<target> [<dependent_target_1> … <dependent_target_n>])

All targets in <dependent_target> list will be brought up to date before <target> is built.

add_executable

Creates a target for building an executable. Basic usage is:

add_executable(<name> [sources])

add_library

Creates a target for building a library. Basic usage is:

add_library(<name> STATIC|SHARED|MODULE [sources])

add_subdirectory

Adds subdirectory to the project. Subdirectory must have its own CMakeLists.txt in it. Effectively it allows you to create sub-modules of the project. However note that in contrast with include() add_subirectory creates its own scope, instead of inheriting parent’s. Basic usage is:

add_subdirectory(<path>)

Path can be absolute or relative.

include_directories

Adds directories to include search paths for the current CMakeLists. Note that include_directories will affect all targets declared after its invocation. Keep it in mind to avoid getting include errors. Basic usage:

include_directories([path_1 … path_n])

Command supports relative paths.

link_directories

Adds directories to library search paths for the current CMakeLists. Note that link_directories will affect all targets declared after its invocation. Keep it in mind to avoid linker errors. Basic usage is:

link_directories([path_1 … path_n])

Command supports relative paths.

target_link_libraries

Allows you to link specific library to the target. Note that generally command is quite complex, but in most basic and common form it looks like:

target_link_libraries(<target_name> [lib_name_1 … lib_name_n])

Library name might be either name or full path to the library. 

CMake: phases and targets

When working with CMake it’s important to remember that it works in two phases and be aware of which phase you’re targeting.

The first phase is the configuration phase itself; it defines how the project is going to be built and usually takes up most of the space of a script(s). However, there is one caveat: CMake script can be (and also often is) executed in the build phase. 

So what is the build phase? The build phase is the process of building targets, defined in the configuration process. 

What are targets? Targets essentially are commands executed by a build system during the build process. It may be a little bit cryptic, so let’s consider how you define target in CMake. Usual ways to define targets would be commands: add_library, add_executable and add_custom_target. I think the first two are self-explanatory and pretty much explain what targets are – it’s a sequence of commands defined to assemble a certain piece of software (library or executable). 

What about add_custom_target then? You can think about it as about a built-in way to extend the build process with custom commands. Where add_library or add_executable will strictly build a library or executable, add_custom_target allows you to specify a custom command to be run during build time. One example of such a command would be a clean target, which usually deletes files left after previous builds. 

The last piece in the targets’ puzzle is the fact that you can define relationships between targets with add_dependecies function. That way you can make sure targets are executed in a specific order. All together this system provides you with a strong and flexible suite to build pretty much anything the way you want.

CMake: built-in modules

A module is essentially a collection of commands (or library if you will), which simplifies certain tasks. You can check the reference for a full list of available modules.

ExternalProject

Pretty much a “must-have” module if you’re integrating any external project into yours, especially if the integrated project is not CMake-based. While default implementation actually expects CMake project, by tweaking a few parameters you can easily (well, I guess easier is a better word) integrate configure-based projects. Essentially, you can think about ExternalProject as about system for managing dependencies (like we have Gradle in Android): it allows you to download, unpack, configure, build and install any project you can get sources of.

Let’s start with really basic examples.

Basics

First of all make sure you have included the module like this:

include(ExternalProject)

Next let’s define project we want to add:

ExternalProject_Add(
    example
    PREFIX example_prefix
    URL https://www.example.com/sources.tar.gz
)

That’s a really barebones example which probably won’t really occur in the real life, but let’s go over some basic properties we used here:

  • The first parameter, which is required, is the project name. It will be used to create target for the external project which you can use later on.
  • PREFIX – is essentially a folder where all operations regarding external project are going to take place. Those operations are: download, configuration, build etc.
  • URL – URL to the sources. Note that while usually some remote endpoint is provided to download the archive with sources, you can actually provide a path to the local archive (or folder, but it requires a few additional tricks).

Using git

Quite often you might want to use a git repository with specific version to build:

ExternalProject_Add(
    example
    PREFIX example_prefix
    GIT_REPOSITORY https://github.com/Examples/git-example.git
    GIT_TAG origin/1.2.3
    GIT_SHALLOW 1
)
  • GIT_REPOSITORY – quite straightforward, just a URL to repo or git@ URL.
  • GIT_TAG – despite its name, it’s not tag necessarily. It can be branch, tag or commit.
  • GIT_SHALLOW – if set, downloads only specific tag, without entire repo history.

Using local folder

In Android environment, which usually builds few architectures you probably would like to avoid additional overhead of downloading and unpacking sources for every architecture:

ExternalProject_Add(
    example
    PREFIX example_prefix
    URL ${CMAKE_CURRENT_SOURCE_DIR}/sources
    DOWNLOAD_NO_EXTRACT 1
)

In this case we supply a path to the local folder with sources. DOWNLOAD_NO_EXTRACT makes ExternalProject skip the extract step and just copy sources folder to the prefix source folder.

Building in local folder

While it might not be a great idea for many projects (mainly because there are usually some left-overs which are left by build and they might conflict with future build, while usually you want to make clean builds), in some cases it might be worth it to at least try it out (for example, if you’re really trying to squeeze out every little bit of performance from build process).

ExternalProject_Add(
	example
	SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/sources
	DOWNLOAD_COMMAND “”
	BUILD_IN_SOURCE 1
)

As you can see we don’t supply PREFIX here, instead we set SOURCE_DIR directly. We also override DOWNLOAD_COMMAND to empty string, so we don’t try to download anything. Setting BUILD_IN_SOURCE prevents ExternalProject from creating additional build folder and builds in SOURCE_DIR directly instead.

Additional notes

The built-in steps that ExternalProject executes: DOWNLOAD, UPDATE, CONFIGURE, BUILD and INSTALL (in this order). You can override any of those command with according *_COMMAND parameter and log any of those with LOG_* 1 parameter.

If you using default behavior which builds project as CMake project you can modify behavior with custom arguments like following:

CMAKE_ARGS -DSHARED:BOOL=ON

Or alternatively you might use CMAKE_CACHE_ARGS to force set cache variables.

CMake and Android

Toolchain file

CMake allows to integrate custom platforms with so called toolchain: file which specifies some behavior and flags specific to platform. On Android this file is located ${NDK_LOCATION}/build/cmake/android.toolchain.cmake or you can find the latest version online here.

If you do anything in CMake in Android I highly recommend to have this file opened all the time, since you probably going to constantly look up variables and other stuff.

Important directories

Another Android specific thing would be building different architectures. In your module folder there should be hidden folder called “cxx”. To get to actual build folder navigate to cxx/cmake/${variantName}/${abi}. This folder is quite important because:

  1. Prefix folders of all external projects you added are going to be located here and you probably want to have those open to read logs in case something fails.
  2. CMake generates build files there. Android is using Ninja so files you’re interested in would be: build.ninja and rules.ninja. You might be interested in those to check how your library is built, which flags are used etc. In general it’s a good source of information in case you stuck and need some insight in how it’s actually works under the hood.
  3. All other files, while being much less useful than prefixes and generated build files, still might be of use in certain use cases.

Another important directory you might want to check out is ${CMAKE_LIBRARY_OUTPUT_DIRECTORY} which by default is located at ${MODULE_PATH}/build/intermediates/cmake/${varianName}/obj/${abi}. This is directory where you want to output your shared libs so build system will automatically pick them up for distribution. I also prefer to put “include” folder here as well since it seems more intuitive for me, but you definitely can you use any other directory you want.

Important variables

Android toolchain file defines many useful variables. Some of them are listed below:

  • CMAKE_LIBRARY_OUTPUT_DIRECTORY – directory where your built artifacts will be outputted to.
  • CMAKE_CURRENT_SOURCE_DIR – directory of current CMake file.
  • ANDROID_TOOLCHAIN_PREFIX – path to the Android built toolchain for current ABI. 
  • ANDROID_TOOLCHAIN_ROOT – root of the toolchain. You usually interested in “bin” directory which has all the build tools.
  • ANDROID_TOOLCHAIN_SUFFIX – suffix for toolchain. Empty on *nix host systems, set to “.exe” for Windows.
  • CMAKE_C_COMPILER – C compiler, set by toolchain file.
  • CMAKE_CXX_COMPILER – CXX compiler, set by toolchain file.
  • ANDROID_ASM_COMPILER – assembler, set by toolchain file.
  • ANDROID_AR – archive tool, set by toolchain file.
  • CMAKE_C_FLAGS – C compiler flags, set by toolchain file.
  • CMAKE_CXX_FLAGS – CXX compiler flags, set by toolchain file.
  • CMAKE_ASM_FLAGS – assembler flags, set by toolchain file.
  • CMAKE_SHARED_LINKER_FLAGS – linker flags, set by toolchain files.
  • ANDROID_LLVM_TRIPLE – target triple (eg. aarch64-none-linux-android21 and yes I know it’s not a triple).
  • CMAKE_SYSTEM_PROCESSOR – target processor (eg. i686).
  • CMAKE_ANDROID_ARCH_ABI – current ABI, set by toolchain file.

Additional notes

This section accumulates some additional advices that hopefully will help out other developers to don’t spend half of a day trying to understand what is going on:

  • Android toolchain sets “-Wl,–fatal-warnings” without ability to turn it off. This flag makes all linker warnings to be treated as errors. While it might be good for library you’re writing, it’s probably not a good idea to have it turned on for external libraries, since you have no control over them. If you get errors logs like “warning treated as error” I recommend removing this flag from the CMAKE_SHARED_LINKER_FLAGS.
  • When integrating configure based projects with ExternalProject you will have to pass all flags (c, cxx, ld and asm flags) manually. Note that flags you receive from toolchain doesn’t have “–target” and “–gcc-toolchain” in them and those required by CLang to properly generate libs (especially if you have assembly files in the library). Make sure to append those flags to all relevant flags (for instance for FFmpeg you want to append those to c, ld and asm flags).
  • Make sure that you’re not only specifying proper target for the compiler, but also provide proper target/host to the library itself. Some libraries have different sets of assembly files for different architectures. Look up –help of configure script to see if there are any flags named “target” or “host” or similar. If those are present make sure to properly specify them, otherwise you might end up with conflict where library provides assembly files for you machine (which is default value of host), but assembler expects files for target (phone).
  • PIC and x86. Android forces PIC from Marshmallow and up, however some libraries (notable FFmpeg) does not provide support for PIC on x86 (reasoning being that the performance would be too bad). If you run in such situation check if you can disable assembly for the library (check for flag in the configure –help).
  • You can pass additional arguments to CMake in gradle by setting property arguments in externalNativeBuild.cmake closure (either in defaultConfig or declaration of your flavor/build type) and you can set specific ABIs you’re building for setting property abiFilters in ndk closure.

References

  1. https://www.selectiveintellect.net/blog/2016/7/29/using-cmake-to-add-third-party-libraries-to-your-project-1
  2. http://kai-wolf.me/cpp/android/cmake/gradle/2019/02/18/working-with-cpp-in-android-effectively/
  3. https://github.com/Kitware/CMake/blob/master/Modules/ExternalProject.cmake
  4. https://trac.ffmpeg.org/ticket/4928

Using Python in Pre-build phase in xCode

Standard

Preamble

Pre- and post-build events is very useful feature of xCode, yet quite underestimated and under-researched (in my opinion). Usually, whole interaction with this feature, in an average project, is to press the button to add new “Run Script” phase and copy-paste script from a library documentation, so it can work (I, for instance, usually integrate R.swift straight away). However, since you can run pretty much any command from “Run Script” phase the range of tasks that can be done from there is much wider.

In this article, I would like to show how to integrate a simple Python script, that will check the code and throw the compiler warning (or error) in case there is a problem.

Important note: this article mainly concentrates on Python script creation and it’s integration, rather then Swift code. In case you have questions related to the Swift code – make sure to ask it in the comment section!

TL; DR:

If you’re not particularly interested in digging into details and/or already have a script to integrate or you just want to know how to throw custom warnings/error in “Run Script” phase of xCode – there is the compressed explanation especially for you:

  • To run Python script from xCode, you need to put following in “Run Script” phase:
python "{script_path}"
  • To throw custom error/warning you need to print it to stdout in the following format: “{path}:{line}: (error|warning): {message}”. The “path” format argument is the path to file to show warning/error at, the “line” argument is the line to show warning/error at, error or warning specify wherever it will be error or warning and “message” argument is the message to be shown.
  • If your script prints error – it must finish with a non-zero exit code. For Python, call sys.exit(code) to finish script with an error code. If you don’t do so, xCode will generate the additional error.
  • In case you would rather just play with code, check out the little app I wrote to illustrate the material in the article: GitHub.

Idea outline

There are plenty of reasons to want to implement additional code checks: starting from searching for TODOs in the code base and finishing with more complex rules to enforce (like forcing test-coverage for example).

In this article, I want to show how to implement the check for the model’s layers. For this purpose, we will integrate simple REST endpoint: XKCD, which returns current webcomic from XKCD Site.

Let’s start with a small dictionary to make sure that we all on the same page here:

  • Model’s layer – an abstract conception, which is there to encapsulate all classes which objects are used within certain parts of an application. For example, DB layer objects are used in operations with database (doesn’t matter wherever it’s CoreData or Realm). Model’s layer doesn’t have an “in code” representation, it’s more of an abstract aggregation.
  • Model’s entity – abstract business model, that describes properties that we want to use. For example, from XKCD we have Webcomic, that we want to operate on. That’s an entity. When we have NetWebcomic it’s representation of an entity within the Net layer.

We will use three model layers for this purpose:

  • Net – objects from this layer are responsible for parsing data from the Internet (API, web scrapping, etc.)
  • DB – database layer, objects are representing the data stored in the local database.
  • App – objects from this layer are used in the business logic (for presenting data in UI, CRUD operations, etc.)

Let’s also define a few assumptions that will make the task much easier, while not breaking practical usability and usefulness of the example:

  1. All entities (in our case only one, Webcomic) must have a representation (class) in every layer.
  2. Representation (class) name of entity in a layer must be in following format: “{prefix}{entity_name}”. So representation of Webcomic in the Net layer will be “NetWebcomic”.
  3. All representations (classes) must be in one folder.

The rule that we want to enforce upon all entities is the following: representation of an entity in every layer should have the same properties (i.e. same names, but they can have different types). If there is a class that has/misses property that others doesn’t/do have – our script must conveniently print a warning or throw an error on project building. The idea behind this rule is to make sure that if there is a change in any layer it’s reflected in other layers as well. That would come in handy if you added a property in one layer, but got distracted and forget to add counterpart properties in others.

That pretty much sums up the outline, let me continue to the implementation details, but first I would like to present the reason why I picked Python over other options.

Why Python?

Programmers who have done things with “Run Script” phase may be wondering about the choice of Python there since usually Bash is used for simple tasks.

It actually doesn’t really matter what you use: you can run any language or program from run script phase. However, Python has the following advantages over other options:

  • It’s there out of the box: OSX comes with Python 2.7 installed, so you can use it without requiring your colleagues to install additional packages/programmes.
  • It’s popular language with a lot of battle-tested libraries and wide community to help you out.

If you feel more comfortable with JS or Ruby or any other language – you can apply things explained in this article to those languages as well (to a certain extent, obviously).

Representations

Let me start off with defining actual classes for layers.

Net

Nothing too special here. Almost direct mapping of JSON from the endpoint, except for “date” property. It’s mapped from “day”, “month” and “year” fields of JSON to one field (comic.date = “(day)-(month)-(year)”).

final class NetWebcomic {
    var title: String?
    var image: String?
    var desc: String?
    var date: String?
}

DB

I used CoreData as database framework, you can use Realm instead. Also, note that type of “date” changed. It is not exactly relevant for us, as long as the property “date” exists in the class.

@objc(DBWebcomic)
final class DBWebcomic: NSManagedObject {
    @NSManaged var title: String?
    @NSManaged var image: String?
    @NSManaged var desc: String?
    @NSManaged var date: Date?
}

App

The class that is used in the business logic of the application.

final class AppWebcomic {
    var title: String?
    var image: URL?
    var desc: String?
    var date: Date?
}

Python script

Now let’s dig into actual Python code that will parse our models’ code and check it. I will split the code into sections to make it easier to understand it. The code is supposed to work with Python 2.7 without any additional libraries required, however, it should be easy to adapt it for Python 3.

Define and parse CLI arguments

parser = argparse.ArgumentParser(description='Check validity of models.')
parser.add_argument('-p', '--path', help='Path to the models directory.', dest='path')
parser.add_argument('-e', '--error', help='Treat as error.', dest='error', action='store_true')

args = parser.parse_args()

Package “argparse” is the built-in solution for parsing arguments from command line. We create the object of ArgumentParser class with the description of what our script is doing.

Next, we’re adding command line arguments:

  • Path – can be supplied via “-p=” or “–path=” and will be in “path” property of an object with parsed arguments.
  • Error – if this flag is set, the script will throw an error instead of a warning in inconsistency is found.

The function “parse_args()” collects all defined arguments into an object (“args” in our case) which contains all arguments in properties, names of which are defined by “dest” argument (so the path will be in args.path).

Parse models

Since we assume that all models are in one folder, we just iterate over files in path supplied by command line argument and parse all “.swift” files.

class_re = re.compile(r'class ([^{:]+?)(: [^\s]+)? {\n([^}]+?})', re.DOTALL)
var_re = re.compile(r'(((\s\s\s\s)|\t)(@NSManaged )?var ([^:]+): ([^\n\s]+)( = ([^\n]+?))?)\n', re.DOTALL)

classes = {}
prefixes = ['App', 'DB', 'Net']

for root, dirs, files in os.walk(args.path):
    for file_name in files:
        path = os.path.join(root, file_name)

        if not file_name.endswith('.swift'):
            continue

        f = codecs.open(path, mode='r', encoding='utf-8')
        content = f.read()
        f.close()

        for class_match in class_re.findall(content):
            pair = extract(class_match[0])
            if pair:
                prefix, name = pair

                if name not in classes:
                    classes[name] = {}

                if prefix not in classes[name]:
                    classes[name][prefix] = {'__path__': path}

                for variable_match in var_re.findall(class_match[2]):
                    classes[name][prefix][variable_match[4]] = {'type': variable_match[5],
                                                                '__line__': line_for_str(variable_match[0], content)}

First, I declared a few variables that will help us to parse data and store the result. Variables: class_re and var_re are precompiled regex patterns. First is for finding classes within the file and second for extracting variables from the body of those classes.

Dictionary “classes” is used as a more convenient way of storing parsed data. The layout of the dictionary looks as following:

class_name:
    prefix:
        "__path__": path (helping variable holding path to the particular file)
        variable_name:
            "__line__": line (line of particular variable)
            "type": type (type of variable)

So, for every entity, we store each representation in “prefix” object, each prefix stores path to particular representation and variables objects. Each variable stores type and line in the source code to throw warning/error at.

Parsing itself is rather trivial: we iterate over files and directories at the path with os.walk, which returns us root, directories and files in the current root (os.walk recursively goes over whole hierarchy at the path). Then we iterate over files and create the full path to every file in the current root (os.walk returns files names, not paths).

Then we make sure that file we’re working on is actually “.swift” file (since there might be files from different languages and storyboard/xib files there, which we’re not interested in) and if it’s not we continue to next file.

If the file is actually Swift file – we’re reading it’s content via codecs package, which supports UTF-8 (in 2.7 version of Python UTF files weren’t supported in default “open” function).

Then we’re parsing the content of the file with class_re regex, which gives us all matched occurrences of classes within the file. Every match contains the name of the class in 0 position and it’s content in 2 position (1 position contains a parent class name or an empty string if there is no parent).

Then we’re trying to parse class name. Remember that we assume that every representation’s name is in the format “{prefix}{entity_name}”, so classes with names that don’t match this format don’t interest us. Extraction is don’t via helper function, which is presented below:

def extract(class_name):
    global prefixes

    for prf in prefixes:
        if class_name.startswith(prf):
            return prf, class_name.split(prf)[1]

    return None

This function returns a tuple of prefix and entity name if class name matched format and None if it didn’t. Note that we’re using global variable “prefixes” from the outer scope. In this way, we can adjust supported prefixes in an easy way.

In the loop, we check if “extract” returned a tuple (not None) and if it did, we’re proceeding to store parsed information.

First, we make sure that “classes” have an entity name in it (we add it if it’s not present).

Then we check for the prefix in the same way and adding “__path__” variable in the initialisation process, so we always have the path to throw warning/error at.

And finally, we parse the content of class with var_re regex. It returns the list of variables. We’re interested in positions 0, 4 and 5. Position 0 has the whole matched variable definition which is used to find line number on source code with helper function “line_for_string” (presented below). Position 4 contains the name of the variable and position 5 contains the type of the variable.

def line_for_str(string, text):
    for (index, line) in enumerate(text.split('\n')):
        if string in line:
            return index + 1

    return None

This function is rather trivial: we’re splitting content into lines, iterating over those lines and returning index + 1 (xCode counts lines from 1, not from 0) if there is a match (or None if there is no match).

That sums up parsing section, let’s move on to the generation of warnings/errors.

Validation and warnings/errors generation

Now, when we have all the necessary data prepared in memory, we can validate it and throw a warning/error if there is an inconsistency.

Let me present whole code first:

had_error = False
for class_name, content in classes.iteritems():
    for first, second in itertools.combinations(content.keys(), 2):
        first_keys = content[first].keys()
        second_keys = content[second].keys()

        fr, sn = first, second
        if len(first_keys) &amp;amp;lt; len(second_keys):
            fr, sn = second, first
            first_keys, second_keys = second_keys, first_keys

        for first_var in first_keys:
            if first_var.startswith('__'):
                continue

            passed = False
            for second_var in second_keys:
                if first_var == second_var:
                    passed = True
                    break

            if not passed:
                had_error = True
                print('{}:{}: {}: Inconsistency found at {}'.format(content[fr]['__path__'],
                                                                    content[fr][first_var]['__line__'],
                                                                    'error' if args.error else 'warning', first_var))

Variable “had_error” is there just to indicate wherever there was at least one inconsistency found in check.

We’re iterating over “classes” getting class name and associated with its dictionary. In second loop we’re generating a unique pair of prefixes with help of build-it itertools package. This ensures that each pair of layers is checked only once, so we won’t do the same work twice (and throw warning/error twice as well).

In the next step, we’re ensuring that first object (first keys) have more or equal amount of keys in it. This ensures that if we have an excess variable it’s going to get spotted.

Then we’re iterating over first keys (the key is the variable name) and if the variable starts with “__” we’re ignoring it (variables in the format “__{name}__” are assumed to be internal variables of the script and are not taken into account). Then we iterate over second keys and check if there is the match of names. If there is a match we’re setting flag “passed” to True and breaking loop. Flag passed indicates if a match was found.

After the loop, if a match weren’t found we actually throwing an error warning. The process is quite easy – we just print out data in following format: “{path}:{line}: (error|warning): {message}”. Yep, that’s it, xCode monitors what script prints to the stdout and, if it matches format, – it conveniently shows error or warning in the UI.

Wraping up

The last step that we need to perform is to exit the script correctly. It’s rather an optional step, however doing so will prevent xCode error from happening. To exit script correctly you need to call:

sys.exit(-1 if had_error and args.error else 0)

Function sys.exit finishes the process with exit code supplied as an argument (in our case -1 if there was an error and we treat inconsistency as error and 0 otherwise).

The reason for this is that xCode will throw an error if “Run Script” phase printed errors, but finished with 0 (normal) exit code.

Calling the script

Now when we have our script ready we need to call it in the “Run Script” phase. Just create new standard “Run Script” phase and put the following line in it:

python "$PROJECT_DIR/python-integration/model_validator.py" "--path=$PROJECT_DIR/python-integration/models"

Note that it assumes that script is called “model_validator.py”, the target named “python-integration” and model layers are located in “models” folder.

PROJECT_DIR variable contains path to the project (note, not to the target).

Let’s run it!

Screen Shot 2018-09-09 at 02.20.51

Seems to be working just nicely!

Test app

Note that I left out details of Swift implementation (like a database, fetching data from API and so on). I think it would be more beneficial to concentrate on the script, as it shows how to parse classes and work with the parsed data. In case you would like to play with test app for yourself and see how it works, check out the GitHub page.

Conclusion

I hope that this article will bring at least some developers to use “Run Phase” scripts feature of xCode to improve their code and workflow. What I’ve described is just a very small part of what is possible.

I wish you good luck and hope to see you in the next articles!

Building FFmpeg 4.0 for Android with Clang

Standard

ffmpeg-featured

Preamble

This article is the continuation of my article about building FFmpeg for Android. If you haven’t read it, please, take a look here: Building FFmpeg for Android, since I will skip some basic stuff that already was described here.

In this article, I will concentrate on the specifics of building FFmpeg with Clang. GCC toolchain inside NDK has been deprecated for quite a while now and it’s highly advised to migrate to Clang (since GCC is not maintained).

Important notes

Performance and size

It’s reported that sometimes Clang tends to produce larger and less-performant binaries (libs) than GCC does. I haven’t noticed significant difference in case of FFmpeg, however if you would link FFmpeg with external library (like libx264 for instance), things might get ugly. Keep that in mind.

Assembler

It seems that both GCC and Clang distributed inside most recent (r17b, at the moment of writing) NDK have troubles with ASM code of FFmpeg, which unfortunately forced me to turn it off. Obviously, it will hurt the performance of library quite a bit, so if performance is critical for you – you might want to fall back to earlier versions of NDK and FFmpeg. However, I should aware you that even with enabled asm time of processing of larger videos with complex filters is still quite long. So if time is crucial for you – consider using MediaCodec instead. Hopefully, I will be able to ship article explaining basic processing in a few weeks.

Development Kit

As with previous article, this one ships along with an update to FFmpeg-Development-Kit, which should ease the process of building FFmpeg for mobile platforms. If you not actually interested in digging into details of building and would rather get ready .so files and template to work on – you can skip rest of the article and proceed directly to the GitHub.

Host OS

While it is technically possible to build FFmpeg on Windows – I highly recommend you to use Linux or OSX as the host system. If you only have Windows machine – consider installing Linux as the second OS or use VM. Building of open-source projects on Windows always tends to be painful and cross-building is usually twice as painful. So unless you have time to deal with all kinds of strange errors happening all around you – consider using *nix based machine for the building process.

Setup

Downloading and extraction

First things first: you will need NDK and FFmpeg. I used r17b and 4.0.2 versions respectively. You’re free to use any versions you would like to, however, it might fail to build. FFmpeg is actively developed and NDK is actively developed as well, so there is always a possibility of conflicting changes that would ultimately result in failed integration. In case you run into such case – fallback to the earlier versions. Download NDK from here and FFmpeg from here. Note that Android Studio allows you to download NDK in SDK Manager, however, I personally prefer to use separate distribution and suggest you do the same.

After you’ve downloaded NDK and FFmpeg, unpack NDK to the disc and make sure to put FFmpeg under NDK/sources directory (so it would be something like android-ndk-r17b/sources/ffmpeg).

Android project

Usually, you want to use libraries directly in some project with JNI. I tend to place such project in the same folder where NDK is located (that allows to automate the building process slightly better). A bit more about this in Building section.

Change in FFmpeg sources

Important note: I highly recommend you to don’t change code or files of FFmpeg unless you absolutely have to. Unfortunately,  I actually had to do few changes manually, those changes are listed and explained below.

libavdevice/v4l2.c

Open this file and find the following line:
int (*ioctl_f)(int fd, unsigned long request, ...);

to

int (*ioctl_f)(int fd, unsigned int request, ...);

This change is required to avoid conflict with ioctl function inside of NDK, which have the different signature.

configure

Open configure script with any text editor and navigate to line #5021 or:
SHFLAGS='-shared -Wl,-soname,$(SLIBNAME)'
inside “android)” case in “case $target_os in” switch.
Change the line to:
SHFLAGS='-shared -soname $(SLIBNAME)'

It has to deal with linker and soname of the library being created. Essentially, it seems that FFmpeg should pass flags to CC (which is Clang in our case), however, it actually passes those flags directly to the linker which causes the linker error. To resolve that issue we fix the flags assuming that they’re passed directly to the linker.

fftools/ffmpeg.c

Navigate to line #4852 or:
exit_program(received_nb_signals ? 255 : main_return_code);
and change it to:
ffmpeg_cleanup(received_nb_signals ? 255 : main_return_code);

FFmpeg is command-line program essentially and it has the tendency to call exit(…) or abort() functions under the hood to close the program and it actually places one call to exit(…) just before the program would end normally with “return main_return_code;”. And if FFmpeg is used as binary everything is totally fine, however, in the case of Android native code calling any of those functions will cause a silent crash of the application with no message. In order to avoid that, we actually just calling the cleanup function, without the call to exit(…).

Building

Clang vs GCC

There are few differences between Clang and GCC. First of all, GCC cross compiler is always a binary specially prepared to target a specific platform. If you look into “toolchains” folder inside NDK you will notice a lot of folders like “$arch-4.9”. Each of those folders contains specific GCC toolchain for a particular architecture. Contrary to GCC, Clang has only one binary, which is used to build any of the supported targets.

Script

I highly recommend you to use Bash script for the building. You, obviously, can enter all commands in terminal directly, however, a script will allow you to iteratively test different configurations and persist your work across multiple days (and you want that, believe me).

Variables

Let’s start with setting up variables:

DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # get script directory
PARENT="$(dirname "${DIR}")" # parent directory
NDK="$(dirname "${PARENT}")" # NDK directory
PROJECT="$(dirname "${NDK}")/{project_name}/app" # place your project name instead of {project_name}
PROJECT_JNI="$PROJECT/jni"
PROJECT_LIBS="$PROJECT/libs"
SYSROOT="$NDK/sysroot" # sysroot for the cross-building
HOST= # place your host system here, like "darwin-x86_64" for OSX
LLVM_TOOLCHAIN="$NDK/toolchains/llvm/prebuilt/$HOST/bin" # location of clang binary

Note that this script expects Android project to be located in the same folder where NDK is located. It’s not a strict requirement, you can use any path you want. If you want to use a custom path make sure to set variable PROJECT to your project “app” folder.

CC and linker flags

Next, let’s define common flags for CC and linker:

CFLAGS="-O3 -fPIC"
LDFLAGS="-lc"

-O3 flag tells the compiler to try to optimise generated code to be faster, however resulting binaries/libraries might be larger in size. If size is crucial, set the flag to -Os, which will optimise for size by the cost of speed.

-fPIC tells the compiler to generate Position Independent Code, which is required by Android security standards (non-fPIC libs will not be loaded on Android 6.0 and higher).

-lc flag is passed to the linker and tells linker to search for libc.

Building function

When I was initially integrating FFmpeg I wrote 6 scripts (5 for every architecture and 1 global). However, after reviewing all the paths and configurations I decided that it will be better to create a function, which will mangle parameters by only a few input arguments.

Let’s start the definition:

function build {
ARCH=$1
LEVEL=$2
CONFIGURATION="--disable-asm --enable-cross-compile --disable-static --disable-programs --disable-doc --enable-shared --enable-protocol=file --enable-pic --enable-small $3"
LIB_FOLDER="lib"

This function expects architecture (as the string, with following values: armeabi-v7a, arm64-v8a, x86, x86_64) as first argument (saved in ARCH variable) and platform level as the second argument (as the string, the value must be in following ranges: 14-19, 21-24, 26-28) saved in LEVEL variable.

CONFIGURATION contains common flags passed to the FFmpeg configure script. All are pretty much self-explanatory.

LIB_FOLDER essentially is a workaround for the strange decision of NDK team to put lib files for all architectures inside “lib” folder except for x86_64 (libraries for this architecture for some reason reside in “lib64” folder).

Handling architectures

Next, let’s handle each particular architecture by creating switch for $ARCH:

case $ARCH in
esac

armeabi-v7a

Let’s start from “armeabi-v7a”:

"armeabi-v7a")
TARGET="arm-linux-androideabi"
CC_FLAGS="-target thumbv7-none-linux-androideabi -mfpu=vfpv3-d16 -mfloat-abi=soft"
LDFLAGS="--fix-cortex-a8 $LDFLAGS"
PLATFORM_ARCH="arm"
TOOLCHAIN_FOLDER=$TARGET
;;

TARGET contains the string of target which is used by NDK in internal paths. It will be different for each architecture.

CC_FLAGS contains flags passed to the CC (Clang in our case). We’re interested in -target flag since it defines target we’re compiling for and specifying the wrong target is going to produce incompatible code and thus build will fail.

LDFLAGS – we’re adding –fix-cortex-a8 as a workaround for the bug in Cortex A8 CPU.

PLATFORM_ARCH – contains the string to create the correct path to the platform for particular architecture.

TOOLCHAIN_FOLDER – is a workaround for x86 architecture toolchain (folder actually called “x86”, while TARGET is “i686-linux-android”, for all other architectures this folder name equals target name).

arm64-v8a

"arm64-v8a")
TARGET="aarch64-linux-android"
CC_FLAGS="-target aarch64-none-linux-android -mfpu=neon -mfloat-abi=soft"
PLATFORM_ARCH="arm64"
CONFIGURATION="$CONFIGURATION --disable-pthreads"
TOOLCHAIN_FOLDER=$TARGET
;;

There is the only thing that is really different from armeabi-v7a and it is appending –disable-pthreads to CONFIGURATION. It’s needed because linker for arm64 does not support pthreads command passed by FFmpeg and it produces broken libraries (with “read” in soname).

x86

"x86")
TARGET="i686-linux-android"
CC_FLAGS="-target i686-none-linux-androideabi -mtune=intel -mssse3 -mfpmath=sse -m32"
PLATFORM_ARCH="x86"
TOOLCHAIN_FOLDER=$PLATFORM_ARCH
;;

Note difference in TOOLCHAIN_FOLDER definition. As mentioned x86 toolchain folder name doesn’t match target name, thus we need to define it to have the correct value.

x86_64

"x86_64")
TARGET="x86_64-linux-android"
CC_FLAGS="-target x86_64-none-linux-androideabi -msse4.2 -mpopcnt -m64 -mtune=intel"
PLATFORM_ARCH="x86_64"
LIB_FOLDER="lib64"
TOOLCHAIN_FOLDER=$PLATFORM_ARCH
;;

Note LIB_FOLDER redefinition. As was mentioned above for some reason only x86_64 have its libs in “lib64” folder, not in “lib” folder as every other architecture.

Building toolchain variables definition

TOOLCHAIN=$NDK/toolchains/$TOOLCHAIN_FOLDER-4.9/prebuilt/$HOST/bin
CC=$LLVM_TOOLCHAIN/clang
CXX=$LLVM_TOOLCHAIN/clang++
AS=$CC
AR=$TOOLCHAIN/$TARGET-ar
LD=$TOOLCHAIN/$TARGET-ld
STRIP=$TOOLCHAIN/$TARGET-strip

TOOLCHAIN is a base path to GCC toolchain (Clang falls back to GCC LD and other components).

CC – path to Clang (C compiler).

CXX – path to Clang++ (C++ compiler).

AS – assembler, we will use Clang as assembler as well.

AR – binary for working with archives (packing/unpacking).

LD – linker.

STRIP – binary for stripping unnecessary symbols.

FFmpeg configure

Let’s finally configure the FFmpeg itself:
PREFIX="android/$ARCH"
./configure --prefix=$PREFIX \
$CONFIGURATION \
--ar=$AR --strip=$STRIP --ld=$LD --cc=$CC --cxx=$CXX --as=$AS \
--target-os=android \
--extra-cflags="$CC_FLAGS -I$SYSROOT/usr/include/$TARGET $CFLAGS" \
--extra-ldflags="-L$NDK/toolchains/$TOOLCHAIN_FOLDER-4.9/prebuilt/$HOST/lib/gcc/$TARGET/4.9.x -L$NDK/platforms/android-$LEVEL/arch-$PLATFORM_ARCH/usr/lib $LDFLAGS" \
--sysroot=$SYSROOT --extra-libs=-lgcc

We start by defining PREFIX variable which will define the destination folder for built libraries (android folder in our case).

We pass include directory in –extra-cflags and we pass actually two lib directories in –extra-ldflags. First one is for libgcc and the second one is platform one, which contains implementations of functions specific to the platform and arch.

Note that -lgcc actually passed to the –extra-libs, which make FFmpeg append it to the end of the building command. It’s a bit strange how it works, because Clang generates code that uses GCC built-in functions, but doesn’t provide implementations for them, which requires us to add libgcc manually.

Building of .so files

In order to avoid possible problems, we’re going to use “make” that is distributed inside the NDK:

$NDK/prebuilt/$HOST/bin/make clean
$NDK/prebuilt/$HOST/bin/make -j2
$NDK/prebuilt/$HOST/bin/make install

Make that usually distributed with *nix systems would work fine in most cases, but as an extra precaution, it’s better to use NDK’s one.

Final touches

There is final step left: we need to set up proper module for the NDK and do actual ndk-build of our JNI to create the final library that will use FFmpeg:

export NDK=$NDK
export ARCH=$ARCH
export PLATFORM="android-$LEVEL"
export NDK_PROJECT_PATH=$PROJECT
yes | cp -rf Android.mk "$PREFIX/Android.mk"
$NDK/ndk-build
if [ ! -d "$PROJECT/out" ]; then
mkdir -p "$PROJECT/out"
fi
yes | cp -rf "$PROJECT_LIBS/$ARCH" "$PROJECT/out/$ARCH"}

First of all, we’re exporting variables to make them global (i.e. accessible in ndk-build and according *.mk files).  Note NDK_PROJECT_PATH variable which is pointing to the project, which will be built.

Then we force copy Android.mk to the configure destination folder (for details check out the previous article and/or GitHub repo).

Then we perform actual ndk-build.

As a final touch, we check if “out” folder exists, create it, if it doesn’t, and then copy built set of libraries from project “libs” folder to the “out” folder.

Running the function

Now, after the function is ready, we just need to call it for every architecture. It’s quite trivial:

build "armeabi-v7a" "14"
build "arm64-v8a" "21"
build "x86_64" "21"
build "x86" "14"

Note that building can take quite some time, so you can make yourself a cup of tea or coffee, while you waiting for it to finish.

Bonus!

This section primarily for the people, who interested in additional details or tricks that might help in the process of FFmpeg integration.

Redirecting stdout and stderr

FFmpeg (as pretty much any library) prints its logs into either stdout or stderr. By default, both of them are redirected to /dev/null, which means that this information is not accessible.

In theory, in particular case of the FFmpeg you can declare custom logging function and set it as primary via “av_log_set_callback”, however, for some reason it was causing SIGSEGV (it seems that somehow strings are losing null-terminator and it was causing memory access error), so I implemented another solution based upon this post: How to use standard output streams for logging in android apps.

I haven’t really changed much, except for the fact that I already had logging header (logjam.h), so I created C file (logjam.c) and exposed function:

int start_logger(const char *app_name);

in the header.

Also, the original post was missing headers, so here are headers for the .c file:

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <pthread.h>

I’m calling this function in:

jint JNI_OnLoad(JavaVM* vm, void* reserved)

to spawn redirection thread at the very start of the FFmpeg usage.

Note that FFmpeg by default logs a lot of information, so you might want to set log level to a lower one.

Diagnosing FFmpeg build errors

When you’re trying to experiment with FFmpeg configuration (to make binary smaller, for example), you usually get a lot of errors and need a lot of trials to make everything work.

In order to diagnose what exactly went wrong in building process, navigate to ffbuild/config.log. This file contains logs of configure process and most of the time you’re going to find the cause (or at least lead to the cause) of the problem here.

Usually, if you were able to successfully build FFmpeg with NDK toolchain, ndk-build itself should not bring problems. However, sometimes you can stumble upon undefined symbols problem. If there is such a problem – make sure to check soname of the library and symbols presence with the readelf tool. Also, make sure that target triple is correct: if you built FFmpeg for the wrong triple – ndk-build will not recognise library files and load them.

Summary

It’s not an easy thing to integrate native code in Android. It always has a plenty of pitfalls and hidden problems, which quite frequently appear only in runtime on specific architecture or even device.

However, I hope that this article will make at least process of building and integrating itself easier, than it was for me.

On this positive note, I wish all readers good luck with Android development and waiting to see you in the next articles!

The Mysterious UITableView Case

Standard

Preambule

We have recently converted our project from Swift 2.3 to Swift 3. And everything was good so far until I didn’t have to create new screen that contained Table View. Pretty basic task, huh? Well, one problem appeared and I literally spent one hour trying to resolve this.

Ok let me get to the description of the case itself.

Case description

Below you can find very basic example of code that illustrates the issue:

import UIKit

class ViewController: UIViewController {
     @IBOutlet weak var tableView: UITableView!

     override func viewDidLoad() {
          super.viewDidLoad()

          let view = UIView(frame: CGRect(x: 0, y: 0, width: 0, height: .min))
          self.tableView.tableHeaderView = view
          self.tableView.tableFooterView = view

          self.tableView.dataSource = self
          self.tableView.delegate = self

          self.tableView.reloadData()
      }
}

extension ViewController: UITableViewDataSource {
    func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -&amp;gt; Int {
        return 5
    }

     func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -&amp;gt; UITableViewCell {
         var cell: UITableViewCell!

         if let newCell = tableView.dequeueReusableCell(withIdentifier: "WHAT IS THE KAPPA") {
              cell = newCell
         } else {
              cell = UITableViewCell.init(style: .default, reuseIdentifier: "WHAT IS THE KAPPA")
         }

         cell.textLabel?.text = "O_O"
         return cell
    }
}

extension ViewController: UITableViewDelegate {
     func tableView(_ tableView: UITableView, heightForRowAt indexPath: IndexPath) -&amp;gt; CGFloat {
        return 44.0
     }
}

Seems to be ok, builds without errors and even without warnings, what can go wrong, huh?

Well, if you actually build and run this code you will get something like this:screen-shot-2017-03-02-at-5-19-47-pm

Erm, where did the Table View go?

Solution

I met this issue and spent like 1 hour debugging and searching, what can cause such problem. Basically, if you will try to debug this code you will see that functions:

    func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -&gt; Int {
        return 5
    }
     func tableView(_ tableView: UITableView, heightForRowAt indexPath: IndexPath) -&gt; CGFloat {
        return 44.0
     }

are actually getting called. However, function:

func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -&gt; UITableViewCell {
         var cell: UITableViewCell!

         if let newCell = tableView.dequeueReusableCell(withIdentifier: "WHAT IS THE KAPPA") {
              cell = newCell
         } else {
              cell = UITableViewCell.init(style: .default, reuseIdentifier: "WHAT IS THE KAPPA")
         }

         cell.textLabel?.text = "O_O"
         return cell
    }

is not fired even one time.

toll_face

Basically, solution came to me in pretty random way. I looked up on the setup of Table View and saw following:

let view = UIView(frame: CGRect(x: 0, y: 0, width: 0, height: .min))
self.tableView.tableHeaderView = view
self.tableView.tableFooterView = view

Seems rather normal, doesn’t it? Well, one thing caught my eye on this – it’s small dirty “.min” at the end of the first line.

I thought: “Wasn’t it changed to something else?”. Fast search in the codebase give that “.min” was changed to “CGFloat.leastNormalMagnitude”. Let’s try to change “.min” to “CGFloat.leastNormalMagnitude” in the line above and run it:
screen-shot-2017-03-02-at-5-35-57-pm

Bingo!

The most confusing thing for me was compiler wasn’t complaining about it at all and it somehow prevented cellForRow being called, while other function were working just fine.

I hope that it might help someone to resolve this quite strange and frustrating issue.

Good luck in bug hunting!

 

 

Hunting down TestFlight bugs

Standard

Preambule

Recently, I went into one of the most annoying issues, that I’ve had on my work so far. Basically, problem was following: we’re about to release new application to the store. For this purpose, we prepared build for TestFlight and submitted it to for the external testing.

And then – KABOOM – testers (multiple of them) reported that the application is crashing literally on the startup. So, user launches the application and it instantly crashes and closes itself.

After almost week of error and trials, I managed to resolve issue. And purpose of this post is to share this experience, so, maybe, someone will resolve own issue quicker then I did.

Getting to work

So, first of all, – make sure that you’re not able to reproduce this crash itself:

  1. Edit scheme -> change configuration to “Release”
  2. Run application
  3. Try to find out crash

Ok, if you was able to reproduce the crash, great, you can fix this and skip rest of this post.

In my case, I wasn’t able to get away so easily.

Well, this didn’t work – what we’re going to do next? We’re going to find out crash logs from TestFlight.

Finding crashlogs from TestFlight

In case if iTunes Connect account is not yours and belongs to your company or client – grab credentials and add this account to your xCode:

  1. Click “Xcode” menu
  2. Choose “Preferences”
  3. Select tab “Accounts”
  4. Click “+” on the bottom
  5. Select “Add Apple ID”
  6. Enter credentials that you’ve grabbed

If everything is alright, you should be able to see crash logs from TestFlight now:

  1. Click menu “Window”
  2. Select “Organizer”
  3. Select tab “Crashes”
  4. Select build that was crashing
  5. Wait a second and list of crashes should appear

Ok, now you should have crashlogs. If you’re lucky enough – they’re already symbolicated for you (i.e. it’s not just addresses in memory, but actual names of functions in code).

If they’re actually not symbolicated you will see something like this:

edited_crash_log

I have removed name of project (since it’s project from my work), but it doesn’t really matter here. As you can see here, instead of function name and line in the code, there are only raw addresses in memory. It’s not symbolicated crash log and if you get something like this – next section will explain how to symbolicate it.

Symbolication

Well, I wasn’t that lucky, so I had to symbolicate it by myself. If you’re also not lucky, here how you can do it:

  1. Go to iTunes Connect
  2. Log in with credentials for application
  3. Click “My Apps”
  4. Select “TestFlight” tab
  5. Select “iOS” on left menu
  6. In list select build that was crashing
  7. Click “Download dSYM”

Ok, now you should have symbols that will help you to symbolicate crashlog. First of all, you have to get raw file of crash. In order to do so, do following:

  1. Return to Organizer to “Crashes” tab
  2. Right click on crash that you want to symbolicate
  3. Choose “Show in Finder”
  4. You should get Finder window, with file selected (extension: “xccrashpoint”)
  5. Right click on it and choose “Show Package Content”
  6. Navigate down on folders until you get to the folder containing “.crash” files

Basically, .crash file is a simple text file that contains all details related to crash. Copy any of this files to your working director. I would suggest to rename it into something more shorter, so it will be easier to write to Terminal. I used “temp.crash”.

Now you have to find matching dSYM for this crash. Open crash log and find following line: “Binary Images”. Right after this line you should have line with your application name in it and something like this “”. This is UUID which you can use to find out which dSYM corresponds to this crashlog. Image below depicts place (inside of black rectangle) where you can find it:

corrected-uuid

Navigate to folder with downloaded from iTunes Connect dSYMs and find here dSYM with UUID that was in crashlog. Note that names of dSYMs are in upper case and are separated by “-” (for image above correct dSYM file have name “503C13BD-BB1A-3885-ADB6-82CE15C90BF8.dSYM”). It might be hard to find out dSYM, but you should do it rather quickly. When you will find it – copy it to the working directory, where you copied .crash file previously. I would again suggest you to give it shorter name – I used “temp” for this.

Now the last step: symbolication itself.

First of all, you should find out exact path to “symbolicatecrash” on your system. You can do it in following way:

  1. Open Terminal
  2. Execute: “cd /Applications/Xcode.app”
  3. Execute: “find . -name symbolicatecrash”

You will get something like this: “./Contents/SharedFrameworks/DVTFoundation.framework/Versions/A/Resources/symbolicatecrash”.
Note that it is relative path. To make it absolute just replace dot at the start with:
“/Applications/Xcode.app”
so you will get:
“/Applications/Xcode.app/Contents/SharedFrameworks/DVTFoundation.framework/Versions/A/Resources/symbolicatecrash”

Note that path may vary for different Xcode’s versions (path above for xCode 8) – so this method is more robust to find out exact path for your version.

Now with that path navigate to your working directory (where .crash and .dSYM files are located) and do following:

  1. First of all run: “export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer”
  2. Then run: “/Applications/Xcode.app/Contents/SharedFrameworks/DVTFoundation.framework/Versions/A/Resources/symbolicatecrash (or path that you get above by “find”) temp.crash temp.dSYM > report.txt”

If everything is ok, after few seconds you should get symbolicated crash log (in file called “report.txt” in this case, you can replace it with any other name). Check it, you are searching for crashed thread. Analyze crash log and try to figure out what exactly went wrong.

Unfortunately, for me it wasn’t enough.

Magic crashed line

After symbolicating of crash log, I realized that it’s pointing to line that is not present in code. So how we are supposed to resolve such issue?

Basically, such problem may be related to optimizations. Solution here is to turn it off:

genious

So, here what we’re going to do:

  1. Open your project settings in xCode (not target, top level)
  2. Open “Build Settings”
  3. Search for “Optimization”
  4. Turn to “None” Optimization Level for code generation (there are two places, for Swift compiler and Apple LLVM)
  5. Prepare new build for TestFlight and submit it
  6. Ask testers to reproduce crash

After testers will confirm that crash happened on new build – go to Organizer once again and find this new crash. Now, it should be way more verbose then in previous version. Symbolicate it and you should get all possible information related to crash.

Basically, turning off optimizations helped me to find out root cause of crash. Hopefully, it will do so for you.

Good luck with bug hunting!

References

  1. Tutorial by Christopher Hale on iOS crash symbolication

Building FFmpeg for Android

Standard

ffmpeg-featured

Preambule

Some time ago, I got task on my work which required processing of video on Android. As you probably aware – Android doesn’t deliver  built-in tool for such task (Ok-ok, there actually is MediaCodec, which, in a way, allows you to perform video processing, but about it in the next post). After some googe’ing, I came to conclusion that FFmpeg ideally fits requirements of the task. Those requirements, by the way, were following: application had to trim video down to 30 seconds, lower its bitrate and crop it to square. And everything in time less then 30 seconds.

“Well, now all we need is to build FFmpeg and link it to the application. Like shooting fish in a barrel!”, – thought me.

“Good luck!”, – said FFmpeg:toll_face

For the next two weeks I was fighting with NDK, with FFmpeg’s configure, with linking and other stuff. It wasn’t easy to pull everything together, but at the end I managed to build all necessary .so files for all architectures, that application might need.

In the next sections, I will try to step-by-step explain how to build FFmpeg for Android, but, first of all, I would like to present developed by me Kit that should make this process much easier.

FFmpeg Development Kit

In order to make things easier, sometime ago I’ve prepared special Kit, which should make process of preparing .so libraries for Android much easier.

You can found this Kit on my GitHub here: FFmpeg Development Kit

All setup is described in Readme – if you don’t really want to know how it’s working inside and just want to get .so as soon as possible – you can skip the rest of this post and go straight to the Kit. If you get any problems with that – make sure to throw comment here or issue on repo.

VideoKit

Another library, prepared by me, that you might find useful: VideoKit, basically, is a result of steps described in this article – it allows you to execute standard FFmpeg commands to process video file.

You can find it here: VideoKit

Getting to work

First of all, you have to decide in which way you want to embed FFmpeg into your application. I know three ways to do so:

  1. Using precompiled binary with FFmpeg and then executing this with Runtime.getRuntime().exec(“command”). Not really clean way, and I would recommend to don’t use this.
  2. Building FFmpeg as .so libraries and then executing it’s main from your own code. Quite clean way, that allows you to add appropriate checks and write JNI. However, note that you will have to write some C/C++ code for JNI (note, that NDK doesn’t have all modern features of C++) and you still basically will execute FFmpeg in command-line style.
  3. Use FFmpeg as library and write completely own filters in pure C. I would say the most cleanest way to embed FFmpeg as well as the most hardest. I would highly recommend to don’t go that way, unless you have 3-4 months of completely free time to dig into documentation and code.

In the rest of this post I will try to explain how to embed FFmpeg with second way, since from my point of view it’s the best way to achieve necessary functionality.

Components

You will need following components:

  • FFmpeg sources (I used FFmpeg 3.2.4)
  • Android NDK (I used 13r-b)
  • Patience

I was able to build FFmpeg on OSX (Sierra) and Ubuntu (12.04). While, theoretically, it should be possible to build FFmpeg in Windows with Cygwin, I highly would recommend to don’t go that way. Even if you don’t have Mac and using only Windows OS for development – consider installing Ubuntu as second system or in virtual environment and build FFmpeg in it. As per my experience, it will avoid many hours of frustration and weird errors happening all around you.

After you get everything downloaded – extract NDK somewhere on the disc. Then put FFmpeg sources under NDK/sources path. Note, this is very important for building your own JNI interface later on.

Dictionary

In next sections few terms may appear, that might not be known to reader, who didn’t work with gcc and building of open source libraries previously, in general.

There is a list of such terms with short explanation:

  • Toolchain – tools that are used for compiling and building of sources.
  • Sysroot – directory in which compiler will search for system headers and libraries.
  • Prefix – directory in which result of building will be written to.
  • Cross-prefix – directory of compiler to be used.
  • ABI – architecture of processor (i.e. x86, arm-v6, armv8-64 and so om).
  • CFLAGS – flags for C-compiler.
  • LDFLAGS – flags for linker. 

Configure

First step for FFmpeg building is configuring it. FFmpeg, thru special script called “configure”, allows you to choose which features you need, for which architecture you’re going and so on.

For example, let me present configuration for armeabi (arm-v5):

./configure –prefix=$(pwd)/android/arm
--cross-prefix=$NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-
--target-os=linux
--cpu=armv5te
--arch=arm
--disable-asm
--enable-armv5te
–disable-stripping
--extra-cflags="-O3 -Wall -pipe -std=c99 -ffast-math -fstrict-aliasing -Werror=strict-aliasing -Wno-psabi -Wa,--noexecstack -DANDROID -DNDEBUG-march=armv5te -mtune=arm9tdmi -msoft-float"
--sysroot=$NDK/platforms/android-14/arch-arm/

It looks a bit messy, but, unfortunately, it how it looks like in real world.

You can run:

./configure -h

to get full list of available components and flags that might be used.

Building .so files

If you configured everything properly, you should be able to run following commands:

make clean
make -j2 (change two to number of cores that you want to use)
make install

Be patient – building may take a while to end. If everything is good, you should find out .so files in folder which was specified in –prefix parameter of configure.

Versioning

On some systems FFmpeg is adding version code to the end of .so file, so you might get something like this at the end:

libavdevice.so.55

While it’s ok for usage on desktop systems like OSX or Ubunty or any other – Android will not accept such libraries. In order to remove versioning you have to edit configure script.

Open configure with any text editor and find following lines:

SLIBNAME_WITH_VERSION= SLIBNAME_WITH_MAJOR=

And change to $(SLIBNAME), so you get:

SLIBNAME_WITH_VERSION='$(SLIBNAME)'
SLIBNAME_WITH_MAJOR='$(SLIBNAME)'

This will turn off versioning and should give you standard .so files.

NDK module

I’m assuming that if you’re reading this, you have prepared all necessary .so files and we can go further.

Before we get to ndk-build we have to define module with .so libraries, that NDK will recognize and will be able to use. Definition of module is actually rather easy part: in top-level catalog (i.e. where include and lib folders are located) you have to add file called Android.mk with following content inside it:

LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE:=	<libname>
LOCAL_SRC_FILES:= lib.so
LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
include $(PREBUILT_SHARED_LIBRARY)

This file will tell NDK from where it have to take headers and library files.

JNI interface

This is last thing you have to do before you actually can run ndk-build and get necessary .so files ready to go.

JNI stands for Java Native Interface and it’s basically a bridge between Java code and native C code. Note that it’s not entirely usual Java and C code – it have to follow certain conventions to make everything work together. Let’s start from Java code.

Java part

It’s pretty usual, beside fact that you have to pay attention to package, in which class is located and also it must contain special functions marked with keyword “native”.

Lets consider class named D that is located in package a.b.c and that have native function called “run”:

public class D {
    public native int run(int loglevel, String[] args);
}

This class is similar to interface in a way that native functions doesn’t require implementation. When you will call this function, implementation in C counterpart of code actually will be executed.

Beside special functions – it’s normal class that might contain arbitrary Java code.

This pretty much it as for Java part. Let me present C part of the code.

C code

Your C code that actually uses FFmpeg must match your Java interface. Basically, for class defined above – C part would look as follow:

#include <jni.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>

JavaVM *sVm = NULL;

int main(int level, int argc, char **argv); //Fast forward for FFmpeg main

jint JNI_OnLoad(JavaVM* vm, void* reserved) {
    sVm = vm;
    return JNI_VERSION_1_6;
}

//Counter part for function "run" in class D
JNIEXPORT jint JNICALL Java_a_b_c_D_run(JNIEnv *env, jobject obj, jint loglevel, jobjectArray args) {

}

Looks a bit scary, but if you look more closer – there is nothing special about it. Function name encodes location of class and function in Java part of code (it’s why you should choose package carefully).

When you call “run” in your Java code, “Java_a_b_c_D_run” actually will be called in C part.

Beside some naming conventions – there is no restrictions on C code as well. You can even use C++, however, I should aware you that support of C++ on Android is not full (it’s partially support C++11 standard, if I remember correctly).

NDK-build

This is pretty much last step and after this you’re free to go. Before you run “ndk-build” command you must provide two files – Android.mk and Application.mk. It’s better to locate those files in the same folder in which your .c files are located.

Android.mk is responsible for defining all paths and pulling everything together. Example is below:

LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := videokit //name of produced lib
LOCAL_LDLIBS := -llog -ljnigraphics -lz -landroid //standard libs
LOCAL_CFLAGS := -Wdeprecated-declarations //cflags
ANDROID_LIB := -landroid
LOCAL_CFLAGS := -I$(NDK)/sources/ffmpeg // include path
LOCAL_SRC_FILES := videokit.c ffmpeg.c ffmpeg_filter.c ffmpeg_opt.c cmdutils.c // source files to compile
LOCAL_SHARED_LIBRARIES := libavformat libavcodec libswscale libavutil libswresample libavfilter libavdevice // linked libraries

include $(BUILD_SHARED_LIBRARY)
$(call import-module,ffmpeg/android/$(CPU)) // path to NDK module relative to NDK/sources/

Application.mk defines general configuration of produced library:

APP_OPTIM := release //optimization level
APP_PLATFORM := $(PLATFORM) //platform level
APP_ABI := $(ABI) //ABI level
NDK_TOOLCHAIN_VERSION=4.9 //Version of toolchain used
APP_PIE := false //If "pie" will be used
APP_STL := stlport_shared // C++ STL version

APP_CFLAGS := -O3 -Wall -pipe \
-ffast-math \
-fstrict-aliasing -Werror=strict-aliasing \
-Wno-psabi -Wa,--noexecstack \
-DANDROID -DNDEBUG // Global c-flags

When both files are prepared and configured – navigate to this folder in command line and run ndk-build. Make sure that NDK folder is added to the PATH.

If everything was configured properly – you should get your library and copied FFmpeg libraries in libs folder.

Libraries loading

After you got everything prepared you must load libraries in memory of your application. First of all, make sure that libraries located in right folder in the project. It have to be in  /src/main/jniLibs//. If you will not put it there – Android system will not be able to locate libraries.

Loading is rather easy step, but it may have some pitfalls in  it. Basically, it may look as follows:

static {
    try {
        System.loadLibrary("avutil");
        System.loadLibrary("swresample");
        System.loadLibrary("avcodec");
        System.loadLibrary("avformat");
        System.loadLibrary("swscale");
        System.loadLibrary("avfilter");
        System.loadLibrary("avdevice");
        System.loadLibrary("videokit");

    } catch (UnsatisfiedLinkError e) {
        e.printStackTrace();
    }

}

Try-catch construction is rather optional. But may safe your application from unexpected crash on new architecture or unexpected architecture.

Important note: order matters. If library will not find its dependency already loaded in memory – it will crash. So you have to load first library with no dependencies, then library that depends on first and so on.

Congratulations

If you survived up to this point – you successfully embedded FFmpeg into your application. I know that it’s hard work and sincerely congratulate you with it.

If you, unfortunately, didn’t achieve this goal – you always can ask for help in the comment section and I will try my best to help you.

References and acknowledgements

  1. Excellent tutorial, unfortunately not updated for recent versions
  2. Thread on forum, explaining version system of FFmpeg