stri

CMake 101

(From https://stackoverflow.com/questions/31037882/whats-the-cmake-syntax-to-set-and-use-variables)

When writing CMake scripts there is a lot you need to know about the syntax and how to use variables in CMake.

The Syntax

Strings using set():

  • set(MyString "Some Text")
  • set(MyStringWithVar "Some other Text: ${MyString}")
  • set(MyStringWithQuot "Some quote: \"${MyStringWithVar}\"")

Or with string():

  • string(APPEND MyStringWithContent " ${MyString}")

Lists using set():

  • set(MyList "a" "b" "c")
  • set(MyList ${MyList} "d")

Or better with list():

  • list(APPEND MyList "a" "b" "c")
  • list(APPEND MyList "d")

Lists of File Names:

  • set(MySourcesList "File.name" "File with Space.name")
  • list(APPEND MySourcesList "File.name" "File with Space.name")
  • add_excutable(MyExeTarget ${MySourcesList})

The Documentation

The Scope or “What value does my variable have?”

First there are the “Normal Variables” and things you need to know about their scope:

  • Normal variables are visible to the CMakeLists.txt they are set in and everything called from there (add_subdirectory()include()macro() and function()).
  • The add_subdirectory() and function() commands are special, because they open-up their own scope.
    • Meaning variables set(...) there are only visible there and they make a copy of all normal variables of the scope level they are called from (called parent scope).
    • So if you are in a sub-directory or a function you can modify an already existing variable in the parent scope with set(... PARENT_SCOPE)
    • You can make use of this e.g. in functions by passing the variable name as a function parameter. An example would be function(xyz _resultVar) is setting set(${_resultVar} 1 PARENT_SCOPE)
  • On the other hand everything you set in include() or macro() scripts will modify variables directly in the scope of where they are called from.

Second there is the “Global Variables Cache”. Things you need to know about the Cache:

  • If no normal variable with the given name is defined in the current scope, CMake will look for a matching Cache entry.
  • Cache values are stored in the CMakeCache.txt file in your binary output directory.
  • The values in the Cache can be modified in CMake’s GUI application before they are generated. Therefore they – in comparison to normal variables – have a type and a docstring. I normally don’t use the GUI so I use set(... CACHE INTERNAL "") to set my global and persistant values.

    Please note that the INTERNAL cache variable type does imply FORCE

  • In a CMake script you can only change existing Cache entries if you use the set(... CACHE ... FORCE) syntax. This behavior is made use of e.g. by CMake itself, because it normally does not force Cache entries itself and therefore you can pre-define it with another value.
  • You can use the command line to set entries in the Cache with the syntax cmake -D var:type=value, just cmake -D var=value or with cmake -C CMakeInitialCache.cmake.
  • You can unset entries in the Cache with unset(... CACHE).

The Cache is global and you can set them virtually anywhere in your CMake scripts. But I would recommend you think twice about where to use Cache variables (they are global and they are persistant). I normally prefer the set_property(GLOBAL PROPERTY ...) and set_property(GLOBAL APPEND PROPERTY ...) syntax to define my own non-persistant global variables.

Variable Pitfalls and “How to debug variable changes?”

To avoid pitfalls you should know the following about variables:

  • Lists in CMake are just strings with semicolons delimiters and therefore the quotation-marks are important
    • set(MyVar a b c) is "a;b;c" and set(MyVar "a b c") is "a b c"
    • The recommendation is that you always use quotation marks with the one exception when you want to give a list as list
    • Generally prefer the list() command for handling lists
  • The whole scope issue described above. Especially it’s recommended to use functions()instead of macros() because you don’t want your local variables to show up in the parent scope.
  • A lot of variables used by CMake are set with the project() and enable_language() calls. So it could get important to set some variables before those commands are used.
  • Environment variables may differ from where CMake generated the make environment and when the the make files are put to use.
    • A change in an environment variable does not re-trigger the generation process.
    • Especially a generated IDE environment may differ from your command line, so it’s recommended to transfer your environment variables into something that is cached.

Sometimes only debugging variables helps. The following may help you:

  • Simply use old printf debugging style by using the message() command. There also some ready to use modules shipped with CMake itself: CMakePrintHelpers.cmakeCMakePrintSystemInformation.cmake
  • Look into CMakeCache.txt file in your binary output directory. This file is even generated if the actual generation of your make environment fails.
  • Use variable_watch() to see where your variables are read/written/removed.
  • Look into the directory properties CACHE_VARIABLES and VARIABLES
  • Call cmake --trace ... to see the CMake’s complete parsing process. That’s sort of the last reserve, because it generates a lot of output.

Special Syntax

  • Environment Variables
    • You can can read $ENV{...} and write set(ENV{...} ...) environment variables
  • Generator Expressions
    • Generator expressions $<...> are only evaluated when CMake’s generator writes the make environment (it comparison to normal variables that are replaced “in-place” by the parser)
    • Very handy e.g. in compiler/linker command lines and in multi-configuration environments
  • References
    • With ${${...}} you can give variable names in a variable and reference its content.
    • Often used when giving a variable name as function/macro parameter.
  • Constant Values (see if() command)
    • With if(MyVariable) you can directly check a variable for true/false (no need here for the enclosing ${...})
    • True if the constant is 1ONYESTRUEY, or a non-zero number.
    • False if the constant is 0OFFNOFALSENIGNORENOTFOUND, the empty string, or ends in the suffix -NOTFOUND.
    • This syntax is often use for something like if (MSVC), but it can be confusing for someone who does not know this syntax shortcut.
  • Recursive substitutions
    • You can construct variable names using variables. After CMake has substituted the variables, it will check again if the result is a variable itself. This is very powerful feature used in CMake itself e.g. as sort of a template set(CMAKE_${lang}_COMPILER ...)
    • But be aware this can give you a headache in if () commands. Here is an example where CMAKE_CXX_COMPILER_ID is "MSVC" and MSVC is "1":
      • if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC") is true, because it evaluates to if ("1" STREQUAL "1")
      • if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC") is false, because it evaluates to if ("MSVC" STREQUAL "1")
      • So the best solution here would be – see above – to directly check for if (MSVC)
    • The good news is that this was fixed in CMake 3.1 with the introduction of policy CMP0054. I would recommend to always set cmake_policy(SET CMP0054 NEW) to “only interpret if()arguments as variables or keywords when unquoted.”
  • The option() command
    • Mainly just cached strings that only can be ON or OFF and they allow some special handling like e.g. dependencies
    • But be aware, don’t mistake the option with the set command. The value given to option is really only the “initial value” (transferred once to the cache during the first configuration step) and is afterwards meant to be changed by the user through CMake’s GUI.
stri

Android NDK console print C/C++ – Chirag Patel

When your Android app has C/C++ code using NDK tools and you want printing output to serial port console, here’s the solution.

// change sharing first
chmod(“/dev/console”, S_IRWXG | S_IRWXO | S_IRWXU);
// open file
int serialFd = open(“/dev/console”, O_RDWR | O_NONBLOCK | O_NDELAY | O_NOCTTY | O_APPEND);
__android_log_print(ANDROID_LOG_DEBUG, “Serial”, “Serial console fd=%d errno=%d(%s)”, serialFd, errno, strerror(errno));
if (serialFd >= 0) {
    write(serialFd, my_buffer, buffer_length);
    close(serialFd);
}

stri

Datacenter tiering

Datacenter tiering

Tier 1:
– 99.671% uptime
– annual 28.8 hrs downtime
– full shutdown for preventive maintenance

Tier 2:
– 99.741% uptime
– annual downtime of 22 hrs
– some redundancy with single path for power, requiring shutdown for preventive maintenance

Tier 3:
– 99.982% uptime
– annual 1.6 hrs downtime
– sufficient redundancy to allow planned maintenance
– at least 13.2 KV power

Tier 4:
– 99.995% uptime
– annual downtime of 0.4 hrs
– multiple paths to power and AC and designed to handle worst case scenario with no critical impact
– at least 26.2 KV power

stri

Network tiering

Network tiering

Tier 4:
– most datacenters
– owns internal network
– pay other networks for IP transit outside the facility

Tier 3:
– regional providers
– build redundancy thru’ redundant POPs (points of presence) outside facility
– pay for IP transit past the POPs

Tier 2:
– national or international footprint
– still pay IP transit to reach some portions of Internet

Tier 1:
– do not pay IP transit
– global presence
– don’t pay other providers for any portion of connectivity

stri

Memory Thrashing compiled by Chirag Patel December 27, 2008

Memory Thrashing compiled by Chirag Patel December 27, 2008
(Ref: Embedded Systems Design June 2008)

Memory threshing is a typical problem that goes unnotices while programming time-critical systems. Translation Look-aside Buffers (TLB) are used for data and instruction cache. There are two main cache-replacement schemes: 1) Least Frequently Used (LFU) and 2) Least Recently Used (LRU). Memory accesses require address translations in modern systems. So, when a page table is is found in an on-chip TLB (a TLB hit), the lookup normally does not do translation. On TLB miss, the processor must look again and must calculate offset to find a byte physically.

When a system has multiple concurrent accesses, operating system schedules time slices for those processes. If we consider 32-entry LRU TLB and 6 processes each using different page memory, at least 12 pages are active at any given time (instruction + data). If every process uses double nested procedures (or jumps between 3 pages) in each time slice, there are 36 active pages. If operating system time slices sequentially, by the time 6th process is reached, 30 pages are accessed. By the end of 6th process time slice, 36 page accesses should have passed. So, the cache manager will discard first 4 accesses (LRU algorithm). When the time comes for the first process, it will have TLB misses for those first 4 page accesses. As the processor continues, it keeps discarding the pages that the next process in sequence will need. This memory thrashing greatly degrades system performance.

To avoid thrashing:
– Long unused variables should be declared absolutely rather than relatively.
– Macros should be expanded.
– Avoid nesting procedure calls whenever possible.
– Minimize number of concurrent tasks.
– Don’t use jumps larger than page size unless absolutely necessary.

stri

double precision comparison in c++ – chirag patel

Checking double precision value for limits is tricky.

Say,

double x = 1.0;
if ( x == 1.0 ) do_something();

 

This comparison can fail as double precision value when converted to hexadecimal (what computer can do only) representation, looses some least significant bits or positions beyond double precision digits. So, in above example if ( x == 0.9999999 ) might have worked depending on compiler and/or CPU. Therefore, in programming, we create functions to check such limits to approximate values.

 

// Used for double type comparasion
typedef union mydouble_ {
    double dbl_val;
    unsigned long un_u32_val[2];
    unsigned char un_u8_val[8];
} mydouble;

 

// Checks double value against close enough limits i.e. min <= val <= max
// Return: -1 => val < min
// Return: 0 => min <= val <= max
// Return: 1 => val > max
int my_double_limit_check( double val, double min, double max )
{
    mydouble dbl;
    mydouble cmp;
    int ret = 0;
 
    if ( val < min )
    {
        dbl.dbl_val = val;
        cmp.dbl_val = min;
        if ( dbl.un_u32_val[0] != cmp.un_u32_val[0] )
            ret = -1;
        else if ( dbl.un_u8_val[4] != cmp.un_u8_val[4] )
            ret = -1;
    }
 
    if ( val > max )
    {
        dbl.dbl_val = val;
        cmp.dbl_val = max;
        if ( dbl.un_u32_val[0] != cmp.un_u32_val[0] )
            ret = 1;
        else if ( dbl.un_u8_val[4] != cmp.un_u8_val[4] )
            ret = 1;
    }
 
    return ret;
}
 
// Checks double value against close enough less value i.e. val < min
// Return: -1 => val < min
// Return: 0 => val >= min
int my_double_less_check( double val, double min )
{
    mydouble dbl;
    mydouble cmp;
    int ret = 0;
 
    if ( val < min )
    {
        dbl.dbl_val = val;
        cmp.dbl_val = min;
        if ( dbl.un_u32_val[0] != cmp.un_u32_val[0] )
            ret = -1;
        else if ( dbl.un_u8_val[4] != cmp.un_u8_val[4] )
            ret = -1;
    }
 
    return ret;
}
 
// Checks double value against close enough greater value i.e. val > max
// Return: 1 => val > max
// Return: 0 => val <= max
int my_double_greater_check( double val, double max )
{
    myouble dbl;
    myouble cmp;
    int ret = 0;
 
    if ( val > max )
    {
        dbl.dbl_val = val;
        cmp.dbl_val = max;
        if ( dbl.un_u32_val[0] != cmp.un_u32_val[0] )
            ret = 1;
        else if ( dbl.un_u8_val[4] != cmp.un_u8_val[4] )
            ret = 1;
    }
 
    return ret;
}
 
// Checks double value against close enough equal value i.e. val == eql
// Return: 0 => val == eql
// Retrun: 1 => val != eql
int my_double_equal_check( double val, double eql )
{
    mydouble dbl;
    mydouble cmp;
    int ret = 0;
 
    if ( val != eql )
    {
        dbl.dbl_val = val;
        cmp.dbl_val = eql;
        if ( dbl.un_u32_val[0] != cmp.un_u32_val[0] )
            ret = 1;
        else if ( dbl.un_u8_val[4] != cmp.un_u8_val[4] )
            ret = 1;
    }
 
    return ret;
}

stri

double precision rounding – chirag patel

Uses my_double_greater_check from http://rutmandal.info/double-precision-comparison-in-cc/

double my_double_round( double val, int multiplier )
{
         double val_adj = val * multiplier;
         if ( my_double_greater_check( val_adj floor( val_adj ), 0.0 ) > 0 )
                      val = round( val_adj ) / multiplier;
         return val;
}

round and floor are C math library <math.h> functions.

E.g.

double rnd_val = my_double_round( 0.000998, 1000);

rnd_val will be 0.001