OpenFst Notes

Configure with all the extensions

./configure \
 --prefix=/where/to/install \ #remove this options for default
 --enable-static \
 --enable-pdt  \
 --enable-far \
 --enable-const-fsts \
 --enable-compact-fsts \
 --enable-lookahead-fsts \
 --enable-ngram-fsts

Registering Custom Arcs

OpenFst has an arc registration mechanism. Here are a few fragments that add support for user-defined semiring found in the Kaldi and OpenGrm toolkits

Kaldi Lattice Weight

Compile the below code fragment using the below command, this assume that the OpenFst headers are on the include path.

g++ -g -O2 \
kaldi-arc.cc -o lattice4-arc.so \
-shared \
-I/path/to/kaldi/src \
-DHAVE_ATLAS -fPIC

After compilation make sure that the directory containing lattice4-arc.so is on the LD_LIBRARY_PATH variable.

#include <fst/const-fst.h>
#include <fst/edit-fst.h>
#include <fst/vector-fst.h>
#include <fst/script/register.h>
#include <fst/script/fstscript.h>
#include <lat/kaldi-lattice.h>

using namespace fst;
using kaldi::LatticeArc;

namespace fst {
namespace script {
  REGISTER_FST(VectorFst, LatticeArc);
  REGISTER_FST(ConstFst, LatticeArc);
  REGISTER_FST(EditFst, LatticeArc);
  REGISTER_FST_CLASSES(LatticeArc);
  REGISTER_FST_OPERATIONS(LatticeArc);
  }
}

OpenGrm Lexicographic Weight

Like the Kaldi example this assumes that the OpenFst and OpenGrm headers installed in the default location or on the compilers include. Then just compile with this command g++ -O2 -g lex-arc.cc -o lex-arc.so -shared -fPIC, and again make sure that lex-arc.so is in the current directory or on the LD_LIBRARY_PATH.

#include <fst/const-fst.h>
#include <fst/edit-fst.h>
#include <fst/vector-fst.h>
#include <fst/script/register.h>
#include <fst/script/fstscript.h>
#include <fst/lexicographic-weight.h>

using namespace fst;

typedef LexicographicArc<TropicalWeight, TropicalWeight> LexArc;

namespace fst {
namespace script {
  REGISTER_FST(VectorFst, LexArc);
  REGISTER_FST(ConstFst, LexArc);
  REGISTER_FST(EditFst, LexArc);
  REGISTER_FST_CLASSES(LexArc);
  REGISTER_FST_OPERATIONS(LexArc);
  }
}

Install 32bit on 64bit system

First make sure all the 32bit libraries are installed:

sudo apt-get install gcc-multilib
sudo apt-get install ia32-libs
sudo apt-get install lib32stdc++6
sudo ln -s /usr/lib32/libstdc++.so.6 /usr/lib32/libstdc++.s

Then configure with these extra arugments:

/configure \
--build=i686-pc-linux-gnu "CFLAGS=-m32" "CXXFLAGS=-m32" "LDFLAGS=-m32" \
--prefix=/path/to/install/to \
--enable-static \
--enable-pdt  \
--enable-far \
--enable-const-fsts \
--enable-compact-fsts \
--enable-lookahead-fsts \
--enable-ngram-fsts

At the moment 32bit binaries still run a bit faster on many applications. 32bit does better when there are lots of pointers, this is because the pointer size is halved compared to 64bit and therefore less cache and bandwidth usage. Numerically code is faster on 64bit due to more registers.

Interestingly I noticed that 32bit OpenFst compiled a lot faster than 64bit.