Merge pull request #125 from digitalocean/develop

Release v1.0.6

.gitignore

@@ -2,5 +2,5 @@
 configuration.py
 .idea
 /*.sh
+!upgrade.sh
 fabfile.py
-

README.md

@@ -25,6 +25,8 @@ Questions? Comments? Please join us on IRC in **#netbox** on **irc.freenode.net*
 
 Please see docs/getting-started.md for instructions on installing NetBox.
 
+To upgrade NetBox, please download the [latest release](https://github.com/digitalocean/netbox/releases) and run `upgrade.sh`.
+
 # Components
 
 NetBox understands all of the physical and logical building blocks that comprise network infrastructure, and the manners in which they are all related.

docs/getting-started.md

@@ -58,14 +58,12 @@ NetBox requires following dependencies:
 * libxml2-dev
 * libxslt1-dev
 * libffi-dev
-* graphviz*
+* graphviz
 
 ```
 # apt-get install python2.7 python-dev git python-pip libxml2-dev libxslt1-dev libffi-dev graphviz
 ```
 
-*graphviz is needed to render topology maps. If you have no need for this feature, graphviz is not required. 
-
 You may opt to install NetBox either from a numbered release or by cloning the master branch of its repository on GitHub.
 
 ### Option A: Download a Release
@@ -354,22 +352,99 @@ At this point, you should be able to connect to the nginx HTTP service at the se
 
 Please keep in mind that the configurations provided here are bare minimums required to get NetBox up and running. You will almost certainly want to make some changes to better suit your production environment.
 
-# Upgrading
+## Let's Encrypt SSL + nginx
+
+To add SSL support to the installation we'll start by installing the arbitrary precision calculator language.
 
-As with the initial installation, you can upgrade NetBox by either downloading the lastest release package or by cloning the `master` branch of the git repository. Several important steps are required before running the new code.
+```
+# sudo apt-get -y bc
+```
 
-First, apply any database migrations that were included with the release. Not all releases include database migrations (in fact, most don't), so don't worry if this command returns "No migrations to apply."
+Next we'll clone Let’s Encrypt in to /opt
 
 ```
-# ./manage.py migrate
+# sudo git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
 ```
 
-Second, collect any static file that have changed into the root static path. As with database migrations, not all releases will include changes to static files.
+To ensure Let's Encrypt can publicly access the directory it needs for certificate validation you'll need to edit `/etc/nginx/sites-available/netbox` and add:
 
 ```
-# ./manage.py collectstatic
+    location /.well-known/ {
+        alias /opt/netbox/netbox/.well-known/;
+        allow all;
+    }
+```
+
+Then restart nginix:
+
+```
+# sudo services nginx restart
 ```
 
+To create the certificate use the following commands ensuring to change `netbox.example.com` to the domain name of the server:
+
+```
+# cd /opt/letsencrypt
+# ./letsencrypt-auto certonly -a webroot --webroot-path=/opt/netbox/netbox/ -d netbox.example.com
+```
+
+If you wish to add support for the `www` prefix you'd use:
+
+```
+# cd /opt/letsencrypt
+# ./letsencrypt-auto certonly -a webroot --webroot-path=/opt/netbox/netbox/ -d netbox.example.com -d www.netbox.example.com
+```
+
+Make sure you have DNS records setup for the hostnames you use and that they resolve back the netbox server.
+
+You will be prompted for your email address to receive notifications about your SSL and then asked to accept the subscriber agreement.
+
+If successful you'll now have four files in `/etc/letsencrypt/live/netbox.example.com` (remember, your hostname is different)
+
+```
+cert.pem
+chain.pem
+fullchain.pem
+privkey.pem
+```
+
+Now edit your nginx configuration `/etc/nginx/sites-available/netbox` and at the top edit to the following:
+
+```
+    #listen 80;
+    #listen [::]80;
+    listen 443;
+    listen [::]443;
+
+    ssl on;
+    ssl_certificate /etc/letsencrypt/live/netbox.example.com/cert.pem;
+    ssl_certificate_key /etc/letsencrypt/live/netbox.example.com/privkey.pem;
+```
+
+If you are not using IPv6 then you do not need `listen [::]443;` The two commented lines are for non-SSL for both IPv4 and IPv6.
+
+Lastly, restart nginx:
+
+```
+# sudo services nginx restart
+```
+
+You should now have netbox running on a SSL protected connection.
+
+# Upgrading
+
+As with the initial installation, you can upgrade NetBox by either downloading the latest release package or by cloning the `master` branch of the git repository. Once the new code is in place, run the upgrade script (which may need to be run as root depending on how your environment is configured).
+
+```
+# ./upgrade.sh
+```
+
+This script:
+
+* Installs or upgrades any new required Python packages
+* Applies any database migrations that were included in the release
+* Collects all static files to be served by the HTTP service
+
 Finally, restart the WSGI service to run the new code. If you followed this guide for the initial installation, this is done using `supervisorctl`:
 
 ```

netbox/dcim/filters.py

@@ -8,6 +8,27 @@ from .models import (
 )
 
 
+class SiteFilter(django_filters.FilterSet):
+    q = django_filters.MethodFilter(
+        action='search',
+        label='Search',
+    )
+
+    class Meta:
+        model = Site
+        fields = ['q', 'name', 'facility', 'asn']
+
+    def search(self, queryset, value):
+        value = value.strip()
+        qs_filter = Q(name__icontains=value) | Q(facility__icontains=value) | Q(physical_address__icontains=value) | \
+            Q(shipping_address__icontains=value)
+        try:
+            qs_filter |= Q(asn=int(value))
+        except ValueError:
+            pass
+        return queryset.filter(qs_filter)
+
+
 class RackGroupFilter(django_filters.FilterSet):
     site_id = django_filters.ModelMultipleChoiceFilter(
         name='site',

netbox/dcim/forms.py

@@ -427,7 +427,7 @@ class DeviceFromCSVForm(forms.ModelForm):
         'invalid_choice': 'Invalid site name.',
     })
     rack_name = forms.CharField()
-    face = forms.ChoiceField(choices=[('Front', 'Front'), ('Rear', 'Rear')])
+    face = forms.CharField(required=False)
 
     class Meta:
         model = Device
@@ -446,7 +446,7 @@ class DeviceFromCSVForm(forms.ModelForm):
             try:
                 self.instance.device_type = DeviceType.objects.get(manufacturer=manufacturer, model=model_name)
             except DeviceType.DoesNotExist:
-                self.add_error('model_name', "Invalid device type ({})".format(model_name))
+                self.add_error('model_name', "Invalid device type ({} {})".format(manufacturer, model_name))
 
         # Validate rack
         if site and rack_name:
@@ -457,11 +457,15 @@ class DeviceFromCSVForm(forms.ModelForm):
 
     def clean_face(self):
         face = self.cleaned_data['face']
-        if face.lower() == 'front':
-            return 0
-        if face.lower() == 'rear':
-            return 1
-        raise forms.ValidationError("Invalid rack face ({})".format(face))
+        if face:
+            try:
+                return {
+                    'front': 0,
+                    'rear': 1,
+                }[face.lower()]
+            except KeyError:
+                raise forms.ValidationError('Invalid rack face ({}); must be "front" or "rear".'.format(face))
+        return face
 
 
 class DeviceImportForm(BulkImportForm, BootstrapMixin):

netbox/dcim/models.py

@@ -568,7 +568,10 @@ class Device(CreatedUpdatedModel):
             raise ValidationError("Must specify rack face with rack position.")
 
         # Validate rack space
-        rack_face = self.face if not self.device_type.is_full_depth else None
+        try:
+            rack_face = self.face if not self.device_type.is_full_depth else None
+        except DeviceType.DoesNotExist:
+            raise ValidationError("Must specify device type.")
         exclude_list = [self.pk] if self.pk else []
         try:
             available_units = self.rack.get_available_units(u_height=self.device_type.u_height, rack_face=rack_face,

netbox/dcim/views.py

@@ -61,6 +61,7 @@ def expand_pattern(string):
 
 class SiteListView(ObjectListView):
     queryset = Site.objects.all()
+    filter = filters.SiteFilter
     table = tables.SiteTable
     template_name = 'dcim/site_list.html'
 

netbox/extras/api/views.py

@@ -1,4 +1,4 @@
-import pydot
+import graphviz
 from rest_framework import generics
 from rest_framework.views import APIView
 import tempfile
@@ -49,32 +49,30 @@ class TopologyMapView(APIView):
         tmap = get_object_or_404(TopologyMap, slug=slug)
 
         # Construct the graph
-        graph = pydot.Dot(graph_type='graph', ranksep='1')
+        graph = graphviz.Graph()
+        graph.graph_attr['ranksep'] = '1'
         for i, device_set in enumerate(tmap.device_sets):
 
-            subgraph = pydot.Subgraph('sg{}'.format(i), rank='same')
+            subgraph = graphviz.Graph(name='sg{}'.format(i))
+            subgraph.graph_attr['rank'] = 'same'
 
             # Add a pseudonode for each device_set to enforce hierarchical layout
-            subgraph.add_node(pydot.Node('set{}'.format(i), shape='none', width='0', label=''))
+            subgraph.node('set{}'.format(i), label='', shape='none', width='0')
             if i:
-                graph.add_edge(pydot.Edge('set{}'.format(i - 1), 'set{}'.format(i), style='invis'))
+                graph.edge('set{}'.format(i - 1), 'set{}'.format(i), style='invis')
 
             # Add each device to the graph
             devices = []
             for query in device_set.split(','):
                 devices += Device.objects.filter(name__regex=query)
             for d in devices:
-                node = pydot.Node(d.name)
-                subgraph.add_node(node)
+                subgraph.node(d.name)
 
             # Add an invisible connection to each successive device in a set to enforce horizontal order
             for j in range(0, len(devices) - 1):
-                edge = pydot.Edge(devices[j].name, devices[j + 1].name)
-                # edge.set('style', 'invis') doesn't seem to work for some reason
-                edge.set_style('invis')
-                subgraph.add_edge(edge)
+                subgraph.edge(devices[j].name, devices[j + 1].name, style='invis')
 
-            graph.add_subgraph(subgraph)
+            graph.subgraph(subgraph)
 
         # Compile list of all devices
         device_superset = Q()
@@ -87,17 +85,14 @@ class TopologyMapView(APIView):
         connections = InterfaceConnection.objects.filter(interface_a__device__in=devices,
                                                          interface_b__device__in=devices)
         for c in connections:
-            edge = pydot.Edge(c.interface_a.device.name, c.interface_b.device.name)
-            graph.add_edge(edge)
+            graph.edge(c.interface_a.device.name, c.interface_b.device.name)
 
-        # Write the image to disk and return
-        topo_file = tempfile.NamedTemporaryFile()
+        # Get the image data and return
         try:
-            graph.write(topo_file.name, format='png')
+            topo_data = graph.pipe(format='png')
         except:
             return HttpResponse("There was an error generating the requested graph. Ensure that the GraphViz "
                                 "executables have been installed correctly.")
-        response = HttpResponse(FileWrapper(topo_file), content_type='image/png')
-        topo_file.close()
+        response = HttpResponse(topo_data, content_type='image/png')
 
         return response

netbox/ipam/forms.py

@@ -13,6 +13,10 @@ from .models import (
 )
 
 
+FORM_PREFIX_STATUS_CHOICES = (('', '---------'),) + PREFIX_STATUS_CHOICES
+FORM_VLAN_STATUS_CHOICES = (('', '---------'),) + VLAN_STATUS_CHOICES
+
+
 #
 # VRFs
 #
@@ -215,6 +219,7 @@ class PrefixBulkEditForm(forms.Form, BootstrapMixin):
     vrf = forms.ModelChoiceField(queryset=VRF.objects.all(), required=False, label='VRF',
                                  help_text="Select the VRF to assign, or check below to remove VRF assignment")
     vrf_global = forms.BooleanField(required=False, label='Set VRF to global')
+    status = forms.ChoiceField(choices=FORM_PREFIX_STATUS_CHOICES, required=False)
     role = forms.ModelChoiceField(queryset=Role.objects.all(), required=False)
     description = forms.CharField(max_length=50, required=False)
 
@@ -444,6 +449,7 @@ class VLANImportForm(BulkImportForm, BootstrapMixin):
 class VLANBulkEditForm(forms.Form, BootstrapMixin):
     pk = forms.ModelMultipleChoiceField(queryset=VLAN.objects.all(), widget=forms.MultipleHiddenInput)
     site = forms.ModelChoiceField(queryset=Site.objects.all(), required=False)
+    status = forms.ChoiceField(choices=FORM_VLAN_STATUS_CHOICES, required=False)
     role = forms.ModelChoiceField(queryset=Role.objects.all(), required=False)
 
 

netbox/netbox/settings.py

@@ -11,6 +11,8 @@ except ImportError:
                                "the documentation.")
 
 
+VERSION = '1.0.6'
+
 # Import local configuration
 for setting in ['ALLOWED_HOSTS', 'DATABASE', 'SECRET_KEY']:
     try:

netbox/templates/500.html

@@ -17,6 +17,8 @@
                 <div class="panel-body">
                     <p>There was a problem with your request. This error has been logged and administrative staff have
                     been notified. Please return to the home page and try again.</p>
+                    <p>If you are responsible for this installation, please consider
+                    <a href="https://github.com/digitalocean/netbox/issues">filing a bug report</a>.</p>
                     <div class="text-right">
                         <a href="/" class="btn btn-primary">Home Page</a>
                     </div>

netbox/templates/_base.html

@@ -237,7 +237,7 @@
 		<div class="container">
             <div class="row">
                 <div class="col-md-4">
-                    <p class="text-muted">{{ settings.HOSTNAME }}</p>
+                    <p class="text-muted">{{ settings.HOSTNAME }} (v{{ settings.VERSION }})</p>
                 </div>
                 <div class="col-md-4 text-center">
                     <p class="text-muted">{% now 'Y-m-d H:i:s T' %}</p>

netbox/templates/dcim/device_import.html

@@ -74,12 +74,12 @@
 				<tr>
 					<td>Face</td>
 					<td>Rack face; front or rear (optional)</td>
-					<td>rear</td>
+					<td>Rear</td>
 				</tr>
 			</tbody>
 		</table>
 		<h4>Example</h4>
-		<pre>rack101_sw1,ToR Switch,Juniper,EX4300-48T,Juniper Junos,CAB00577291,Ashburn-VA,R101,21,rear</pre>
+		<pre>rack101_sw1,ToR Switch,Juniper,EX4300-48T,Juniper Junos,CAB00577291,Ashburn-VA,R101,21,Rear</pre>
 	</div>
 </div>
 {% endblock %}

netbox/templates/dcim/site.html

@@ -6,6 +6,26 @@
 {% block title %}{{ site }}{% endblock %}
 
 {% block content %}
+<div class="row">
+    <div class="col-md-9">
+        <ol class="breadcrumb">
+            <li><a href="{% url 'dcim:site_list' %}">Sites</a></li>
+            <li>{{ site }}</li>
+        </ol>
+    </div>
+    <div class="col-md-3">
+        <form action="{% url 'dcim:site_list' %}" method="get">
+            <div class="input-group">
+                <input type="text" name="q" class="form-control" placeholder="Search" />
+                <span class="input-group-btn">
+                    <button type="submit" class="btn btn-primary">
+                        <span class="glyphicon glyphicon-search" aria-hidden="true"></span>
+                    </button>
+                </span>
+            </div>
+        </form>
+    </div>
+</div>
 <div class="pull-right">
     <button type="button" class="btn btn-primary" data-toggle="modal" data-target="#graphs_modal" data-obj="{{ site.name }}" data-url="{% url 'dcim-api:site_graphs' pk=site.pk %}" title="Show graphs">
         <i class="glyphicon glyphicon-signal" aria-hidden="true"></i>

netbox/templates/dcim/site_list.html

@@ -14,5 +14,28 @@
     {% include 'inc/export_button.html' with obj_type='sites' %}
 </div>
 <h1>Sites</h1>
-{% render_table table 'table.html' %}
+<div class="row">
+	<div class="col-md-9">
+        {% render_table table 'table.html' %}
+    </div>
+    <div class="col-md-3">
+		<div class="panel panel-default">
+			<div class="panel-heading">
+				<strong>Search</strong>
+			</div>
+			<div class="panel-body">
+				<form action="{% url 'dcim:site_list' %}" method="get">
+					<div class="input-group">
+						<input type="text" name="q" class="form-control" placeholder="Name" {% if request.GET.q %}value="{{ request.GET.q }}" {% endif %}/>
+						<span class="input-group-btn">
+							<button type="submit" class="btn btn-primary">
+								<span class="glyphicon glyphicon-search" aria-hidden="true"></span>
+							</button>
+						</span>
+					</div>
+				</form>
+			</div>
+		</div>
+    </div>
+</div>
 {% endblock %}

netbox/utilities/forms.py

@@ -253,6 +253,9 @@ class BulkImportForm(forms.Form):
             else:
                 for field, errors in obj_form.errors.items():
                     for e in errors:
-                        self.add_error('csv', "Record {} ({}): {}".format(i, field, e))
+                        if field == '__all__':
+                            self.add_error('csv', "Record {}: {}".format(i, e))
+                        else:
+                            self.add_error('csv', "Record {} ({}): {}".format(i, field, e))
 
         self.cleaned_data['csv'] = obj_list

requirements.txt

@@ -5,6 +5,7 @@ django-filter==0.13.0
 django-rest-swagger==0.3.7
 django-tables2==1.2.1
 djangorestframework==3.3.3
+graphviz==0.4.10
 Markdown==2.6.6
 ncclient==0.4.7
 netaddr==0.7.18
@@ -12,6 +13,5 @@ paramiko==2.0.0
 psycopg2==2.6.1
 py-gfm==0.1.3
 pycrypto==2.6.1
-pydot==1.0.2
 sqlparse==0.1.19
 xmltodict==0.10.2

upgrade.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+# This script will prepare NetBox to run after the code has been upgraded to
+# its most recent release.
+#
+# Once the script completes, remember to restart the WSGI service (e.g.
+# gunicorn or uWSGI).
+
+# Install any new Python packages
+pip install -r requirements.txt --upgrade
+
+# Apply any database migrations
+./netbox/manage.py migrate
+
+# Collect static files
+./netbox/manage.py collectstatic --noinput