Requires.io is all about being notified of outdated dependencies on PyPI packages.

Today we are proud to introduce Pipenv support!

Requires.io will now look for Pipenv and Pipenv.lock files at the root of your repository to warn you about oudated and insecure dependencies, licenses, changelogs... and even issue pull requests to update your Pipenv.

We hope you will enjoy this new feature!

I run a small tea subscription service called Tomotcha, and each month we send Japanese tea directly from Osaka.

Like most small businesses we use Google tools extensively (Analytics, AdWords, Webmaster Tools...) and we periodically receive their emails about new stuffs.

Last week's topic was Does the speed of your mobile website makes you lose some visitors? (loosely translated from French).

Interesting.

I ran it and... it was bad.

Eleven seconds! Try by yourself on your favorite url here.

A friend of mine based in US told me lately that our website was a little bit slow (our servers are in Paris).

I already started to check CloudFront. But with eleven seconds, the problem was obviously deeper than the server location.

And so started my optimization journey on a pretty common stack (ours): Nginx & Django.

Django

I started with Django. We had dozens of CSS and JS files, it was time to consolidate them.

Django Compressor is the way to go.

Edit your settings.py file and modify it as follows after pip installing the django_compressor package:

INSTALLED_APPS = [
    # ...
    'compressor',
    # ...
]
STATICFILES_FINDERS = [
    # ...
    'compressor.finders.CompressorFinder',
]

In your templates, group a maximum of CSS links and JavaScript scripts between a minimum number of compress instructions:

{% load compress %}
<!-- ... -->
{% compress css %}
<link rel="stylesheet" href="{% static 'startup/flat-ui/bootstrap/css/bootstrap.css' %}">
<!-- ... -->
<link rel="stylesheet" href="{% static 'tomotcha/css/style.css' %}">
{% endcompress %}
<!-- ... -->
{% compress js %}
<script src="{% static 'tomotcha/js/underscore-min.js' %}"></script>
<!-- ... -->
<script src="{% static 'js/angular.min.js' %}"></script>
{% endcompress %}

This will aggregate your various files in a single one, with network gains in perspective.

The icing on the cake: set COMPRESS_OFFLINE = True in settings.py and issue a manage.py compress command to avoid on the fly cache generation and latency for your first visitor.

Nginx

You should definitively check that gzip is enabled and actually compress some stuff.

Use Chrome inspection tool and look for the Content-Encoding header in the Response Headers.

If not, you should have a look at the following settings in your Nginx configuration:

server {
    [...]
    gzip on;
    gzip_comp_level    5;
    gzip_min_length    256;
    gzip_types *;
    [...]
}

Do not forget to restart it.

Stripe

Stripe is our payment provider. Awesome service with a great APIs.

We have very basic needs, so we only use Stripe Checkout.

On Tomotcha the checkout button is on the home page and its integration was done pretty much like this:

<script src="https://checkout.stripe.com/checkout.js"></script>
<script type="text/javascript">
(function() {
    [...]
    StripeCheckout.configure({
        key: 'tomotcha',
        [...]
    });
})();
</script>

Don't. Look at the Performance tab of Chrome inspection tool. Catastrophic:

Instead, call StripeCheckout.configure when your customer actually wants to subscribe.

Its a little longer but a nice wheel make them wait, time to display the form.

Resources

A vast subject:

• Do I really need to include my theme's JS for a fancy carousel that I do not use? No: drop it!
• Is my header image small enough? Initially at 225k, optimized for the web it's now 78k: use the dedicated feature in your favorite image editor.
• Check that you include minified (*.min.js) JavaScript files, at least for external dependencies.
• ...

Time to collect. Let's re-run the benchmark:

Better! And now we can start to think about advanced stuff:

• Server location
• Caching
• ...

My conclusion? Coffee is deprecated, upgrade to green tea.

More seriously, it only took me a couple hours to greatly improve our website performance, and it's definitively worth a shot.

DomainKeys Identified Mail (DKIM) and Sender Policy Framework (SPF) are now enabled on all Requires.io emails to ensure their delivery.

In addition, our email notification system is now registered and compliant with Gmail Email Markup so as to provide an easy access to your project's page:

Hope you'll enjoy these improvements!

Since version 8.0 (released a week ago), pip can check downloaded package archives against local hashes to protect against remote tampering. To verify a package against one or more hashes, add the hashes at the end of your requirements.txt files:

FooProject==1.2 --hash:sha256=2cf24dba5fb0a30e26e83b2ac5b9e29e \
                --hash:sha256=486ea46224d1bb4fb680f34f7c9ad96a

Requires.io is now compatible with this feature, as you can see for instance on this project: mozilla/kuma.

A big thank to jezdez for notifying us of this new feature of pip!

Requires.io is proud to introduce a new feature: license tracking for your requirements!

And if a licence differs between your version of a package and its latest one, you get the information in both Requirement and Latest columns.

We hope you will enjoy this new feature!

We released today a new version of the ShiningPanda plugin for Jenkins. For those who don't know this plugin yet, it is probably the easiest way to setup your Python projects with Jenkins. But let's have a look at the new features:

Coverage 4

The Publish coverage.py HTML reports publisher is now compatible with coverage>=4.

Virtualenv

The plugin now bundles the lastest version (13.1.2) of the virtual Python environment builder.

As you may know, requires.io is all about being notified of outdated dependencies on PyPI packages.

We just released a new event-driven notification system, rewritten from scratch to bring more flexibility to all requires.io users.

Email and pull-request policies are now configured at the organization level within the Hooks section of your dashboard.

Email and Pull Request (for GitHub) are available, and Web Hook should be available very soon.

You can of course specify some hook specific information like a comma separated list of email addresses.

But you can also fine-tune the targets: repositories, branches, tags or sites. For these fields, use a comma separated list of glob patterns.

Note the special case of the @default branch: this will target master, default or whatever default branch you defined on GitHub or Bitbucket.

This new system is very much event-driven. For each dependency status (up-to-date, outdated, insecure), you can select when to be notified or not.

In the above example, if your project's status:

• is up-to-date: you will never be notified
• becomes outdated: you will be notified
• stays outdated: you will not be notified
• is insecure: you will always be notified (on new package version, on commits modifying requirements.txt etc...)

Periodic digests are also available: by opting in you will get a daily, weekly or monthly digest regardless of the project's status.

A question? Drop us an email!

requires.io has a new logo!

This logo was designed with lots of dedication by the very talented Ayaka Yamamoto. She is now designing a new logo for our Japanese tea subscription service: tomotcha.com. The current logo looks like a cup of coffee...

Ayaka is based in Japan, but speaks very good English (and even quite a bit of French), so do not hesitate to contact her, or to check her online portfolio.

Over the last few weeks we've released a number of small patches and bugfix improving the overall experience with requires.io, to keep on helping you keep track of Python dependencies.

Handling of prereleases

To figure out if a package is up-to-date, requires.io was using a very simple strategy: take the latest [1] version available on pypi, and match it against the requirement. If it matched, the requirement was up-to-date, otherwise it was outdated.

This strategy falls short for two edge cases: prereleases and private forks.

requests==dev    # Master fetched from GitHub
django==1.7.4.1  # A private fork of Django

These edge cases are now correctly handled, and such dependencies are simply flagged up-to-date.

[1] It was (and still is) a tad more complicated as we distinguish between unstable and stable releases, but this is beyond the scope of this blog post.

Compatible requirements and pull-requests

A lot of projects on GitHub are using requires.io pull-requests. But until now we didn't handle "compatible" requirements correctly.

django>=1.6,<1.7

Such a requirement is now updated to:

django>=1.7,<1.8

Don't forget that you can specify directives in your requirements files to discard updates you don't need. For instance the following requirement would be flagged up-to-date and not yield any pull-request, despite being `outdated`:

# Outdated bug flagged `up-to-date`
django>=1.6,<1.7  # rq.filter:<1.7

Requires.io uses badges generated by shields.io. Today (February 1st, 2015) shields.io is switching its default badges to a sleek, flat design. The rational is that it will be a better match for GitHub overall design.

For consistency purpose, we have decided to make the switch:

You can of course still use the former style (called "plastic") by appending a GET parameter to badges requests, like this: &style=plastic.

As you may know, requires.io is all about being notified of outdated dependencies on PyPI packages.

Therefore we are a privileged observer of the Python community, and it was high time we release some data!

We'll share some more in the upcoming weeks, but let's start with the most popular packages expressed as the percentage of repositories (followed by requires.io) that depends on them:

The result will not come as a surprise: Django is number one with almost 33% of the projects depending on it. We were also quite happy to notice that quality related tools (like coverage, mock, flake8, etc.) rank quite high on the list, evidence that the Python community cares a lot about code quality!

To package maintainers: not on the list? Send us an email and we'll share figures with you.

Stay tuned!

Today requires.io introduces Site Monitoring, a security feature to check that the dependencies of the Python apps deployed on your production servers are up-to-date and secure.

Requires.io can already monitor the requirements of your projects from their source code. We expanded the API so that by adding two lines to your deployment scripts you can now check that your production apps are secure:

$ pip install -U requires.io
$ requires.io update-site -t $MY_SECRET_TOKEN -r $MY_REPO

Step-by-step Tutorial

In this small tutorial we will setup Site Monitoring for the project requires/myapp. This tutorial assumes that you already have an account on requires.io... If you don't, just register!

1. Plan upgrade

First ensure that your plan support the Site Monitoring feature. This can be done from the settings page. In this case I need an Indie+ account.

2. Upgrade your deployment script

Go to the "monitoring" section of your settings. There you can just copy the necessary line. In this case it is:

requires.io update-site -t 6ade5eb345d8a79ad69a9f868021e0210522aceb -r REPO

The token is valid for the account requires, so for the project requires/myapp we just need to replace REPO by myapp.

requires.io update-site -t 62717a87341c8500d316bf52635a9e40ced04ace -r myapp

For an app deployed with a simple fabric script (using fabtools to handle the virtualenv), the resulting script would look similar to this:

with fabtools.python.virtualenv(virtualenv):
    run('pip install -r requirements.txt')
    run('pip install requires.io')
    run('requires.io update-site -t 6ade5eb345d8a79ad69a9f868021e0210522aceb -r myapp')

Adapt for your own deployment scripts!

4. Check the result

Just go to your requirements page on requires.io: you will see a new section called "Sites" in the right column.

Notifications

Notifications for the Site Monitoring feature are coming very soon... Requires.io notification system is being thoroughly updated, but it is not quite ready yet.

Heroku

We are currently testing the requires.io Heroku app. So if you want to hook requires.io to your heroku account to use the Site Monitoring feature, let us know!

Requires.io introduces new beautiful badges, generated thanks to Shields.io:

To switch to the new svg badges just replace .svg by .png in your badge urls.

PS: We are looking for testers to tryout our new Heroku app. If you want to get notified when a requirement on your production system is compromised, drop us an email at info@requires.io.

Requires.io helps you keep track of the requirements of your Python projects.

Today we are introducing an API to push your dependency files.

It's very simple to use, and this is definitively the way to go if you are not using GitHub or Bitbucket.

Get started in 4 steps:

1. Sign Up for an API account
2. Get your API token
3. Install the requires.io package from PyPI
4. Call requires.io on the command line

Typical use case in a build or deployment script would be like:

$ pip install -U requires.io
$ requires.io -a $API_TOKEN -r $REPO_NAME /path/to/my/repo

On the pricing side:

• $0 per month with unlimited public repositories for open source projects
• $15 per month for unlimited private repositories with our Enterprise plan

So register now!