coin

The COIN project

Coin is Illyse's Information System, designed to manage both members and Internet accesses, such as through DSL, VPN, wireless…

It is written in Django, and features a generic configuration interface, which allows to implement custom backends for different technologies. Currently implemented is a LDAP-based backend for OpenVPN, and a very simple DSL backend, without any authentication (useful for "white label" DSL).

Coin currently only works with python2, because python-ldap is (as of 2013) not compatible with python3.

The project page (issue, wiki, etc) is here:

https://code.ffdn.org/FFDN/coin/

A mailing list is available, for both users and developers: https://listes.illyse.org/wws/info/si

License

Coin is released under the GNU Affero GPL 3.0 license. See LICENSE for details.

Extending Coin

If you want to write your own backend, see EXTENDING.md.

Quickstart

Virtualenv

Using a virtualenv is recommended. On Debian, install python-virtualenv. On Archlinux, the package is called python2-virtualenv, and you must replace the virtualenv command with virtualenv2 in the following.

To create the virtualenv (the first time):

virtualenv ./venv

To activate the virtualenv (you need to do this each time you work on the project):

source ./venv/bin/activate

Install dependencies. On Debian, you will probably need the python-dev, python-pip, libldap-dev, libpq-dev, libsasl2-dev, and libjpeg-dev packages.

sudo apt-get install python-dev python-pip libldap2-dev libpq-dev libsasl2-dev libjpeg-dev libxml2-dev libxslt1-dev libffi-dev python-cairo libpango1.0-0

You need a recent pip for the installation of dependencies to work. If you don't meet that requirement (Ubuntu trusty does not), run:

pip install "pip>=1.5.6"

In any case, you then need to install coin python dependencies:

pip install -r requirements.txt

You should now be able to run python manage.py (within the virtualenv, obviously) without error.

Settings

The coin/settings_local.py file is ignored by Git: feel free to override any setting by writing into that file. For example, to override the DEBUG settings:

echo '# -*- coding: utf-8 -*-' > coin/settings_local.py
echo 'DEBUG = TEMPLATE_DEBUG = True' >> coin/settings_local.py

See the end of this README for a reference of available configuration settings.

Database

At this point, you should setup your database. You have two options.

With PostgreSQL (for developpement), recomended

The official database for coin is postgresql.

To ease developpement, a postgresql virtual-machine recipe is provided through vagrant.

Note: Vagrant is intended for developpement only and is totaly unsafe for a production setup.

Install requirements:

sudo apt install virtualbox vagrant

Then, to boot and configure your dev VM:

vagrant up

Default settings target that vagrant+postgreSQL setup, so, you don't have to change any setting.

With SQLite

SQLite setup may be simpler, but some features will not be available, namely:

• automatic allocation of IP subnets (needs proper subnet implementation in the database)
• sending automated emails to remind of expiring membership fee (needs aggregation on date fields, see Django doc)

To use sqlite instead of PostgreSQL, you have to override local settings with someting like:

DATABASES = {
    # Base de donnée du SI
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': 'coin.sqlite3',
        'USER': '', # Not needed for SQLite
        'PASSWORD': '', # Not needed for SQLite
        'HOST': '',  # Empty for localhost through domain sockets
        'PORT': '',  # Empty for default
    },
}

For both PostgreSQL and SQLite

The first time, you need to create the database, create a superuser, and import some base data to play with:

python manage.py migrate
python manage.py createsuperuser
python manage.py loaddata offers ip_pool # skip this if you don't use PostgreSQL

Note that the superuser will be inserted into the LDAP backend exactly in the same way as all other members, so you should use a real account (not just admin/admin).

Then, at each code update, you will only need to update dependencies and apply new migrations:

pip install -r requirements.txt
python manage.py migrate

At this point, Django should run correctly:

python manage.py runserver

Running tests

There is a set of unit tests you can run with :

DJANGO_SETTINGS_MODULE=coin.settings_test ./manage.py test

LDAP-related tests are disabled by default.

Setup LDAP parameters and activate LDAP in settings to make the LDAP tests run.

With pytest

Setup:

pip install pytest-django

Run:

pytest

Models graph

*You need to have graphviz package installed on your system*.

You can build a huge PDF of the models used in coin with :

./manage.py graph_models \
  coin members offers billing resources reverse_dns configuration isp_database \
  -o models.pdf \
  --group-models

You can tweak the app list to graph (eg: if you use optional apps).

For example, if you want to graph only the hardware_provisioning app, run :

./manage.py graph_models \
  hardware_provisioning
  -o models.pdf \
  --group-models

And for more fine tunning of graph generation :

./manage.py graph_models --help

Available commands

Some useful administration commands are available via manage.py.

python manage.py members_email: returns email addresses of all members, one per line. This may be useful to automatically feed a mailing list software. Note that membership is based on the status field of users, not on membership fees. That is, even if a member has forgot to renew his or her membership fee, his or her address will still show up in this list. More filters are available, see the command's help for more details.

python manage.py charge_subscriptions: generate invoices (including a PDF version) for each subscriber. You probably want to run this command every month as a cron task, see below.

python manage.py call_for_membership_fees: send reminder emails to members whose membership fee is about to expire or is already expired (1 month before, on the day of expiration, 1 month after, 2 months after, and 3 months after). You should run this command in a cron job every day.

python manage.py offer_subscriptions_count: Returns subscription count grouped by offer type.

python manage.py import_payments_from_csv: Import a CSV from a bank and match payments with services and/or members. At the moment, this is quite specific to ARN workflow

Configuration

You should first setup the sites application, in the admin. The domain name configured there is used for outgoing emails.

LDAP

By default, LDAP support is disabled. If you want to use LDAP (for instance for the OpenVPN/LDAP backend, or for your own backend), see the configuration file coin/settings_local.example-illyse.py.

ISP-specific configuration

The main place to customize Coin for your ISP is the IspDatabase application, in the admin. Information entered in this application has two purposes:

1) generate a JSON file usable by http://db.ffdn.org/

2) customize the user interface (postal address, bank account, ISP name & logo...)

Some bits of configuration are done in settings.py: LDAP branches, RSS feeds to display on the home page, and so on. Lastly, it is possible to override all Coin templates (i.e. for user views and emails), see below.

Sending emails

Coin sends mails, you might want to customize the sender address:

• DEFAULT_FROM_EMAIL app-wide setting
• administrative email field, from ISPDatabase app, if set, will take precedence over DEFAULT_FROM_EMAIL for administrative emails (welcome email and membership fees), will take precedence (if filled).

Cron tasks

You may want to run cron jobs for repetitive tasks.

The command that generates invoices should be run every day, so that invoices for new subscriptions are created automatically. On the first day of each month, invoices will be generated for all subscriptions. This will run the command at 23:59 every day:

59 23 * * * /home/coin/venv/bin/python manage.py charge_subscriptions`

To send reminder emails for membership fee expiration:

42 3 * * * /home/coin/venv/bin/python manage.py call_for_membership_fees`

If you want to be notified by email each time a reminder email is sent, you can set an email address in the crontab:

MAILTO=tresorier@myisp.fr

Customizing templates

You may want to override some of the (HTML or email) templates to better suit your structure needs.

Coin allows you to have a folder of custom templates that will contain your templates, which gets loaded prior to coin builtins.

With this method, several templates can be overridden:

• template files in coin/templates/
• template files in coin/<app_name>/templates/ for all active applications

Do once (setup)

• Create a folder dedicated to your custom templates (hint: outside of coin git tree is better).
• Register that folder in the EXTRA_TEMPLATE_DIRS settings.

For instance, in settings_local.py:

EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',)

For each template you want to override

Copy the template you want to override to the right place in your custom folder (that's the hard part, see the example).

Example

Say we want to override the template located at coin/members/templates/members/emails/call_for_membership_fees.html and we set EXTRA_TEMPLATE_DIRS = ('/home/coin/my-folder/templates',) in settings.

Then make a copy of the template file (and customize it) at /home/coin/my-folder/templates/members/emails/call_for_membership_fees.html

Good to go :-)

Using optional apps

Some apps are not enabled by default :

• vpn: Management of OpenVPN subscription and credentials through LDAP
• simple_dsl: Simple DSL subscriptions, without handling any authentication backend or user configuration ("marque blanche")
• hardware_provisioning : Self-service app to manage hardware inventory, hardware lent to members or in different stock sites.
• maillists : Self-service mailling-list (un)subscription for members: handles subscribers list for a third-party mailling-list server (sympa, mailman… etc). See maillists app documentation (fr).

You can enable them using the EXTRA_INSTALLED_APPS setting. E.g. in settings_local.py:

EXTRA_INSTALLED_APPS = (
    'vpn',
)

If you enable an extra-app after initial installation, make sure to sync database :

./manage.py migrate

nb: extra apps are loaded after the builtin apps.

Settings

List of available settings in your settings_local.py file.

• EXTRA_INSTALLED_APPS: See using optional apps
• EXTRA_TEMPLATE_DIRS: See customizing templates
• LDAP_ACTIVATE: See LDAP
• MEMBER_MEMBERSHIP_INFO_URL: Link to a page with information on how to become a member or pay the membership fee
• MEMBERSHIP_FEE_REMINDER_DATES: how long before/after the membership fee anniversary date we want to send a reminder email to the member. It defaults to the following:

MEMBERSHIP_FEE_REMINDER_DATES = [
    {'months': -3},  # 3 months before
    {'months': -2},  # 2 months before
    {'months': -1},  # 1 month before
    {'days': 0},     # the day of anniversary
    {'months': +1},  # 1 month after
]

• SUBSCRIPTION_REFERENCE: Pattern used to display a unique reference for any subscription. Helpful for bank wire transfer identification
• REGISTRATION_OPEN : Allow visitor to join the association by register on COIN
• ACCOUNT_ACTIVATION_DAYS : All account with unvalidated email will be deleted after X days
• MEMBERSHIP_REFERENCE : Template string to display the label the member should indicates for the bank transfer, default: "ADH-{{ user.pk }}"
• DEFAULT_MEMBERSHIP_FEE : Default membership fee, if you have a more complex membership fees policy, you could overwrite templates
• PAYMENT_DELAY: Payment delay in days for issued invoices ( default is 30 days which is the default in french law)
• MEMBER_CAN_EDIT_PROFILE: Allow members to edit their profiles
• HANDLE_BALANCE: Allows to handle money balances for members (False default)
• INVOICES_INCLUDE_CONFIG_COMMENTS: Add comment related to a subscription configuration when generating invoices
• MEMBER_CAN_EDIT_VPN_CONF: Allow members to edit some part of their vpn configuration
• MAILLIST_SYNC_COMMAND : The command to send the list of mail addresses of a given mailling list to mail list server. The command will receives one address/line on stdin. This setting could use placholders:
  • {email}: the mail address of the list
  • {short_name}: the list name
• DEBUG : Enable debug for development do not use in production : display stracktraces and enable django-debug-toolbar.
• SITE_TITLE: the base of site title (displayed in browser window/tab title)
• SITE_HEADER: the site header (displayed in all pages as page header)
• SITE_LOGO_URL: URL to website logo (by default, this is an ascii-duck)
• SUBSCRIPTIONS_NOTIFICATION_EMAILS : Emails on which to send notifications uppon new registration.

Accounting logs

To log 'accounting-related operations' (creation/update of invoice, payment and member balance) to a specific file, add the following to settings_local.py :

from settings_base import *
LOGGING["formatters"]["verbose"] = {'format': "%(asctime)s - %(name)s - %(levelname)s - %(message)s"}
LOGGING["handlers"]["coin_accounting"] = {
    'level':'INFO',
    'class':'logging.handlers.RotatingFileHandler',
    'formatter': 'verbose',
    'filename': '/var/log/coin/accounting.log',
    'maxBytes': 1024*1024*15, # 15MB
    'backupCount': 10,
}
LOGGING["loggers"]["coin.billing"]["handlers"] = [ 'coin_accounting' ]

More information

For the rest of the setup (database, LDAP), see https://doc.illyse.net/projects/ils-si/wiki/Mise_en_place_environnement_de_dev

For real production deployment, see file DEPLOYMENT.md.