Migrating Odoo

Odoo Migration

Use https://markdowntohtml.com/

Note: This process does not utilise Python virtualenv which seems to be the best practice for this sort of deployment


These commands taking place on “host” where the production instance is running
The production instance is consistently named “odoo11-live” (linux user, /opt location, filestore, DB name, DB user)

Stop Odoo and ensure it is not running

root@host:~# service odoo11-live stop
root@host:~# ps awxf | grep odo
 4458 pts/1    S+     0:00          \_ grep --color=auto odo

## Confirm name of database to backup

root@host:~# grep db_name /etc/odoo11-live.conf
db_name = odoo11-live

## Check DB exists

root@host:~# su – postgres
postgres@host:~$ psql
psql (9.5.12)
Type “help” for help.

postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
acuto | acuto | UTF8 | en_US.UTF-8 | en_US.UTF-8 | acuto=CTc/acuto
odoo11-dev | odoo11-dev | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
odoo11-live | odoo11-live | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
odoo11live | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | odoo11=CTc/postgres
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(7 rows)

postgres=# \q

## Backup the DB -O or --no-owner for setting no owner on dump.

postgres@host:~$ pg_dump -O odoo11-live | gzip > 20180430-odoo11-live.sql.gz
postgres@host:~$ logout

## Move the backup file to /root

root@host:~# mv /var/lib/postgresql/20180430-odoo11-live.sql.gz .

## Backup the Odoo application

root@host:~# cd /opt
root@host:/opt# tar czf 20180430-odoo11-live.tar.gz odoo11-live

## Move the backup file to /root

root@host:/opt# cd
root@host:~# mv /opt/20180430-odoo11-live.tar.gz .

## Start Odoo again

root@host:~# service odoo11-live start

# Restore These commands taking place on “dev” where the development instance is to be restored The development instance is consistently named “odoo11-dev” (linux user, /opt location, filestore, DB name, DB user, Apache VirtualHost) ## Copy the backup files from the production host

root@dev:~# cd /opt
root@dev:/opt# scp root@host:20180430* .
20180430-odoo11-live.sql.gz 100% 2416KB 2.4MB/s 00:00
20180430-odoo11-live.tar.gz 100% 407MB 135.7MB/s 00:03

## Extract application & rename

root@dev:/opt# tar xf 20180430-odoo11-live.tar.gz
root@dev:/opt# mv odoo11-live odoo11-dev

## Prepare linux user Add Linux user (set a password), change ownership of application directory

root@dev:/opt# adduser odoo11-dev –home /opt/odoo11-dev
root@dev:/opt# chown -R odoo11-dev:odoo11-dev odoo11-dev

## Rename the Odoo filestore The Odoo filestore must match the name of the Odoo database.

root@dev:/opt# cd odoo11-dev/.local/share/Odoo/filestore/
root@dev:/opt/odoo11-dev/.local/share/Odoo/filestore# mv odoo11-live odoo11-dev
root@test:/opt/odoo11-dev/.local/share/Odoo/filestore# cd /opt

## Create the log directory

root@dev:/opt# mkdir /var/log/odoo11-dev
root@dev:/opt# chown odoo11-dev:odoo11-dev /var/log/odoo11-dev

## Install PostgreSQL

root@dev:/opt# apt update
root@dev:/opt# apt upgrade
root@dev:/opt# apt install postgresql -y

## Create the DB User and the DB and import

root@dev:/opt# su – postgres
postgres@dev:~$ createuser -d -P odoo11-dev # Password must match that in /etc/odoo11-dev.conf

postgres@dev:~$ su – odoo11-dev # Use the password you set above

odoo11-dev@dev:~$ createdb odoo11-dev
odoo11-dev@dev:~$ gunzip -c /opt/20180430-odoo11-live.sql.gz | psql odoo11-dev
odoo11-dev@dev:~$ logout
postgres@test:~$ logout

## Install Python requirements

root@dev:~# apt-get install python3 python3-pip
root@dev:~# apt-get install wget git bzr python-pip gdebi-core -y
root@dev:~# apt-get install python-pypdf2 python-dateutil python-feedparser python-ldap python-libxslt1 python-lxml python-mako python-openid python-psycopg2 python-pybabel python-pychart python-pydot python-pyparsing python-reportlab python-simplejson python-tz python-vatnumber python-vobject python-webdav python-werkzeug python-xlwt python-yaml python-zsi python-docutils python-psutil python-mock python-unittest2 python-jinja2 python-pypdf python-decorator python-requests python-passlib python-pil -y
root@dev:~# pip3 install pypdf2 Babel passlib Werkzeug decorator python-dateutil pyyaml psycopg2 psutil html2text docutils lxml pillow reportlab ninja2 requests gdata XlsxWriter vobject python-openid pyparsing pydot mock mako Jinja2 ebaysdk feedparser xlwt psycogreen suds-jurko pytz pyusb greenlet xlrd 
root@dev:~# apt-get install python3-suds -y
root@dev:~# apt-get install node-clean-css -y
root@dev:~# apt-get install node-less -y
root@dev:~# apt-get install python-gevent -y
root@dev:~# pip3 install phonenumbers
root@dev:~# pip3 install gevent

## Install wkhtmltopdf

root@dev:/opt# wget https://downloads.wkhtmltopdf.org/0.12/0.12.1/wkhtmltox-0.12.1_linux-trusty-amd64.deb
root@test:/opt# dpkg -i wkhtmltox-0.12.1_linux-trusty-amd64.deb

## Prepare Config

root@dev:/opt# scp root@host:/etc/odoo11-live.conf .
root@dev:/opt# cat odoo11-live.conf | sed ‘s/odoo11-live/odoo11-dev/g’ > odoo11-dev.conf
root@dev:/opt# mv odoo11-dev.conf /etc

### Sample

admin_passwd = snip
db_host = localhost
db_port = 5432
db_user = odoo11-dev
db_password = odoo
db_name = odoo11-dev
addons_path = /opt/odoo11-dev/server/addons,/opt/odoo11-dev/custom_addons
logfile = /var/log/odoo11-dev/odoo11-dev.log
xmlrpc_port = 8089
data_dir = /opt/odoo11-dev/.local/share/Odoo/

## Prepare Init Script

root@dev:~# scp root@host:/etc/systemd/system/odoo11-live.service .
root@dev:~# cat odoo11-live.service | sed ‘s/odoo11-live/odoo11-dev/g’ > odoo11-dev.service
root@dev:~# mv odoo11-dev.service /etc/systemd/system/

### Inspect Inspect the init script and ensure: * python3 path is valid; and * user and group are correct ie. odoo-dev in this case ### Sample


ExecStart=/opt/odoo11-dev/odoo11-venv/bin/python3 /opt/odoo11-dev/server/odoo-bin -c /etc/odoo11-dev.conf


## Test Start Odoo and Monitor Log

root@dev:~# su – odoo11-dev
odoo11-dev@dev:~$ python3 /opt/odoo11-dev/server/odoo-bin -c /etc/odoo11-dev.conf

Control-C to shut down

odoo11-dev@dev:~$ logout

## Test Init Script

root@dev:~# systemctl daemon-reload
root@dev:~# service odoo11-dev start
root@dev:~# service odoo11-dev status
* odoo11-dev.service – odoo11-dev
Loaded: loaded (/etc/systemd/system/odoo11-dev.service; disabled; vendor preset: enabled)
Active: active (running) since Mon 2018-04-30 10:36:49 UTC; 3s ago

## Enable Start on Boot

root@dev:/etc/systemd/system# systemctl enable odoo11-dev.service
Created symlink from /etc/systemd/system/multi-user.target.wants/odoo11-dev.service to /lib/systemd/system/odoo11-dev.service.

## Install Apache

root@dev:~# apt install apache2 -y
root@dev:~# a2enmod proxy_http
root@dev:~# a2enmod ssl
root@dev:~# a2enmod rewrite

## Prepare LetsEncrypt for SSL

root@dev:~# apt install software-properties-common -y
root@dev:~# add-apt-repository ppa:certbot/certbot
root@dev:~# apt update
root@dev:~# apt install python-certbot-apache -y
root@dev:~# certbot –apache certonly

(Provide customer’s email address – domain owner)

## Apache Virtual Host configuration Create the site in /etc/apache2/sites-available/odoo11-dev.conf This configuration was created for “outsourceitsupport.com”. Search and replace all instances of outsourceitsupport.com with whatever hostname is being used for the migrated Odoo instance.

DocumentRoot /var/www/html
ServerName outsourceitsupport.com

    RewriteEngine on
    RewriteCond %{SERVER_NAME} =outsourceitsupport.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

    ErrorLog ${APACHE_LOG_DIR}/outsourceitsupport.com.error.log
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/outsourceitsupport.com.access.log combined

DocumentRoot /var/www/html
ServerName outsourceitsupport.com

    ProxyPass /        http://localhost:8089/ retry=0
    ProxyPassReverse / http://localhost:8089/ retry=0

    SSLCertificateFile /etc/letsencrypt/live/outsourceitsupport.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/outsourceitsupport.com/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf

    ErrorLog ${APACHE_LOG_DIR}/outsourceitsupport.com-ssl.error.log
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/outsourceitsupport.com-ssl.access.log combined

## Enable the config and reload Apache

root@dev:/etc/apache2/sites-available# ln -s /etc/apache2/sites-available/odoo11-dev.conf /etc/apache2/sites-enabled/
root@dev:/etc/apache2/sites-available# service apache2 reload

Minimal Ubuntu Time Machine Backup Service

I’ve been using a Time Machine backup over AFP to an openmediavault (OMV) box. I’ve got some new hardware and installed Ubuntu 16.04 instead. Alas, OMV uses a custom-rolled version of netatalk 3.x to provide AFP services, while Ubuntu (and Debian upstream) only provide 2.x.

This seemed strange to me. netatalk 3.0 was released over 5 years ago, and should have become available in Debian long ago, so I started to research why there was no netatalk 3.x package in Debian. Along the way, I bumped into a comment on a Debian bug report which mentioned that Apple is phasing out the AFP protocol in favour of SMB (note the comment at the very end of article). Ah!

Right, so, I still have no idea why netatalk 3.x is not available in Debian, but it made perfect sense to me to explore what had been done in the Samba project to implement support for Time Machine.

As luck would have it, Samba 4.8.0 was released a couple of weeks ago, and is the first version to include the Time Machine backup support. Unfortunately, because it is so new, it is not be available as an installable package in Ubuntu 16.04 or even 18.04. So, to use it, one must compile Samba from source.

This process works for Ubuntu 16.04 and I am now using it instead of OMV for Time Machine backups.

The steps below will produce a Time Machine backup service that is open and accessible to anyone on the network (ie. guest only). If you want a permissioned service, you will need to adjust the Samba global configuration and the configuration of the Time Machine share to your preferred permissioning method.

Perform the following steps as the root user.  This is easiest via interactive sudo:

sudo -i

Install pre-requisites:

apt-get install -y libreadline-dev git build-essential \
libattr1-dev libblkid-dev autoconf python-dev \
python-dnspython libacl1-dev gdb pkg-config libpopt-dev \
libldap2-dev dnsutils acl attr libbsd-dev docbook-xsl \
libcups2-dev libgnutls28-dev tracker libtracker-sparql-1.0-dev \
libpam0g-dev libavahi-client-dev libavahi-common-dev \
bison flex avahi-daemon avahi-discover avahi-utils libnss-mdns \

Download the source, configure, build and install:

cd /usr/src
wget https://download.samba.org/pub/samba/stable/samba-4.8.0.tar.gz
tar -xzvf samba-4.8.0.tar.gz
cd samba-4.8.0
./configure --systemd-install-services --with-shared-modules=idmap_ad --enable-debug --enable-selftest --with-systemd --enable-spotlight --prefix=/opt/samba4
make install

Because we installed it to /opt/samba4, add this to the path:

echo 'export PATH=$PATH:/opt/samba4/bin:/opt/samba4/sbin' >> /etc/profile
source /etc/profile

Create the log file location:

mkdir -p /var/log/samba

Check that the version we are using includes Avahi, Spotlight and Time Machine (check that the flags returned by each of the 3 commands match):

# smbd -b | grep -i avahi
# smbd -b | grep -i spotlight
# smbd -b | grep -i fruit

Below is a minimal /opt/samba4/etc/smb.conf that has been carefully checked to ensure that parameters that are on by default or implied by others have been removed. This configuration assumes that the location you want to save Time Machine backups to is mounted at /mnt/backup.  We used a RAID 1 mirror formatted with ext4.


map to guest = Bad User
vfs objects = catia fruit streams_xattr
log file = /var/log/samba/%m
log level = 2
mdns name = mdns

# Time Machine
fruit:veto_appledouble = no
fruit:encoding = native
fruit:metadata = stream

# Security
server min protocol = SMB2

[Time Machine]
fruit:time machine = yes
path = /mnt/backup
guest only = yes
writeable = yes

Starting Samba:

# smbd -s /opt/samba4/etc/smb.conf

To perform your backup, select the “Time Machine” on your server from the “Available Disks” in Time Machine and get started.



Validating South African Cell Numbers with a Regular Expression (Regex)

These days legislation increasingly places an obligation on processors of personal information to ensure that information is accurate.  Input sanitisation and data validation are critical approaches to maintaining accuracy.

I scrounged information about active South African mobile number prefixes using information from the Wikipedia article on South African telephone numbers, a February 2016 post on the Asterisk.org.za mailing list, as well as HLR validation logs.

After munging all of the info from those sources, the following regular expression – as at today – validates all cellphone numbers using known active prefixes in South Africa and excludes the rest.

I took care to try and ensure the regex is as character compact as possible while still providing for all known 3 and 4 digit prefixes.

The regex is:


An example usage (including start and ending terminators) would be:

SELECT * FROM cellnumber_listing
WHERE NOT (trim(celltelephone) ~ '^0((60[3-9]|64[0-5]|66[0-5])\d{6}|(7[1-4689]|6[1-3]|8[1-4])\d{7})$');

Or for Perl Compatible Regular Expression (PCRE):

grep -P '0((60[3-9]|64[0-5]|66[0-5])\d{6}|(7[1-4689]|6[1-3]|8[1-4])\d{7})' file.txt

If you have any comments on the above or suggestions how it could be improved, please comment!

Ramblings on Residential RAID

For the average home NAS, Read/Write performance is not usually a critical factor.

Space efficiency is often the main reason why people with 3 or more drives decide to use something other than RAID 1 (mirroring). But, IMHO for the average – less technical – home user, anything more than RAID1 is asking for trouble.

The biggest benefit of RAID1 is simplicity. One drive mirrors the other. 4 x 4TB drives = 2 x 4TB storage in 2 x 4TB RAID 1 mirrors.

If a drive fails, replace it and the mirror will rebuild. Do it ASAP because you want the rebuild to happen before the sole remaining drive can fail.

If you want to treat the array as 1 x 8TB device instead of 2 x 4TB, you can look at RAID10 which benefits from RAID1 and stripes the two RAID1 arrays into a single device.

However, I would again suggest KISS. One of the biggest risks with a home NAS is that when a problem finally occurs, it has been AGES since you built the thing, and you don’t remember the config exactly – what goes where, what to do to rebuild etc.

         RAID1                   RAID10
     4TB   |   4TB                8TB
    RAID1  |  RAID1         RAID1     RAID1
    (4TB)  |  (4TB)         (4TB)     (4TB)
    (4TB)  |  (4TB)         (4TB)     (4TB)

Read more


The Anatomy of an RSA ID Number

Every South African citizen or resident is issued with a 13 digit ID number in the format YYMMDD GSSS CAZ, where:

  • YYMMDD is the date of birth. Note this means that a person born on 1 Jan 1900 and 1 Jan 2000 will have the same first 6 digits (000101);
  • G indicates gender, where females are assigned sequential numbers in the range 0-4 and males from 5-9;
  • SSS is a sequence number of the birth registered on that birth date;
  • C indicates citizenship, where 0 is a SA citizen, and 1 is a permanent resident (only citizens can vote);
  • A is usually an 8 (could also be a 9 according to DHA [1]). Prior to 1986 [2] this number was used to indicate the holder’s race;
  • Z is a checksum digit.

The checksum digit (Z) is calculated using the Luhn algorithm :
A = The sum of the odd-positioned digits (positions 1,3,5,7,9,11 and excluding 13/Z)
B = The concatenation of the even-positioned digits (positions 2,4,6,8,10,12)
C = The sum of the digits in the result of B x 2
D = A + C
Z = 10 – (D mod 10) [3]

Thus, for a theoretical ID number of a South African Resident female born on 22 November 2000, with ID number 001122 3344 182, the calculation of the check bit in red is as follows:

A = 0 + 1 + 2 + 3 + 4 + 1 = 11
B = 012348
C = 012348 x 2 = 24696
C = 2 + 4 + 6 + 9 + 6 = 27
D = 11 + 27 = 38
Z = 10 - (38 % 10) = 10 - 8 = 2

Note also, that the fictitious ID number 001122 3344 182 is valid for a birth date of 22 November 1900 as well as 22 November 2000.

An excellent spreadsheet by Robert MacLean which implements this check for your reuse is available from here.

And that’s that.

1. Note in the new ID books a note on this bit says, “Usually 8. If yours is a 9 please ensure that you have a letter confirming authentication of your ID documents from Home Affairs.”

2. Identification Act No 72 of 1986 repealed the 1952 Blacks (Abolition of Passes and Co-ordination of Documents) Act and large portions of the 1950 Population Registration Act. Identity numbers would no longer reflect a person’s race group in terms of the 1950 Population Registration Act or any other law. Influx control regulations were lifted and passes were to be replaced by a uniform identity document for all population groups.

3. The mod or Modulo operator as used here effectively returns the rightmost digit of the number on the left. If the result of the mod operation is a 0, then Z turns out to be 10 – 0 = 10. In this instance the the resulting check bit is 0.

NOTE: To validate South African ID number using Javascript, try this.