Designing Websites for iPhone X

We've already covered "The Notch" and the options for dealing with it from an HTML and CSS perspective. There is a bit more detail available now, straight from the horse's mouth:

Safe area insets are not a replacement for margins.

... we want to specify that our padding should be the default padding or the safe area inset, whichever is greater. This can be achieved with the brand-new CSS functions min() and max() which will be available in a future Safari Technology Preview release.

@supports(padding: max(0px)) {
    .post {
        padding-left: max(12px, constant(safe-area-inset-left));
        padding-right: max(12px, constant(safe-area-inset-right));
    }
}

It is important to use @supports to feature-detect min and max, because they are not supported everywhere, and due to CSS’s treatment of invalid variables, to not specify a variable inside your @supports query.

Jeremey Keith's hot takes have been especially tasty, like:

You could add a bunch of proprietary CSS that Apple just pulled out of their ass.

Or you could make sure to set a background colour on your body element.

I recommend the latter.

And:

This could be a one-word article: don’t.

More specifically, don’t design websites for any specific device.

Although if this pushes support forward for min() and max() as generic functions, that's cool.

Direct Link to Article — Permalink

---

Designing Websites for iPhone X is a post from CSS-Tricks

from CSS-Tricks http://ift.tt/2wMqglP
via IFTTT

Exploring Animation And Interaction Techniques With WebGL (A Case Study)

Marvin Visions

Marvin Visions is a new typeface designed in the spirit of those letters you’d see in scruffy old 80's sci-fi books. This specimen site has a really beautiful layout that's worth exploring and reading about the design process behind the work.

Direct Link to Article — Permalink

---

Marvin Visions is a post from CSS-Tricks

from CSS-Tricks http://ift.tt/2xpCvsM
via IFTTT

The Importance Of JavaScript Abstractions When Working With Remote Data

Recently I had the experience of reviewing a project and accessing its scalability and maintainability. There were a few bad practices here and there, a few strange pieces of code with lack of meaningful comments. Nothing uncommon for a relatively big (legacy) codebase, right?

However was something that I keep finding. A pattern that repeated itself throughout this codebase and a number of other projects I've looked through.They could be all by lack of abstraction. Ultimately, this was the cause for maintenance difficulty.

• In object-oriented programming, abstraction is one of the three central principles (along with encapsulation and inheritance). Abstraction is valuable for two key reasons:
• Abstraction hides certain details and only show the essential features of the object. It tries to reduce and factor out details so that the developer can focus on a few concepts at a time. This approach improves understandability as well as maintainability of the code.
• Abstraction helps us to reduce code duplication. Abstraction provides ways of dealing with crosscutting concerns and enables us to avoid tightly coupled code.

The lack of abstraction inevitably leads to problems with maintainability.

Often I've seen colleagues that want to take a step further towards more maintainable code, but they struggle to figure out and implement fundamental abstractions. Therefore, in this article, I'll share a few useful abstractions I use for the most common thing in the web world: working with remote data.

It's important to mention that, just like everything in the JavaScript world, there are tons of ways and different approaches how to implement a similar concept. I'll share my approach, but feel free to upgrade it or to tweak it based on your own needs. Or even better - improve it and share it in the comments below! ❤️

API Abstraction

I haven't had a project which doesn't use an external API to receive and send data in a while. That's usually one of the first and fundamental abstractions I define. I try to store as much API related configuration and settings there like:

• the API base url
• the request headers:
• the global error handling logic

  const API = {
    /**
     * Simple service for generating different HTTP codes. Useful for
     * testing how your own scripts deal with varying responses.
     */
    url: 'http://httpstat.us/',
  
    /**
     * fetch() will only reject a promise if the user is offline,
     * or some unlikely networking error occurs, such a DNS lookup failure.
     * However, there is a simple `ok` flag that indicates
     * whether an HTTP response's status code is in the successful range.
     */
    _handleError(_res) {
        return _res.ok ? _res : Promise.reject(_res.statusText);
    },
  
    /**
     * Get abstraction.
     * @return {Promise}
     */
    get(_endpoint) {
        return window.fetch(this.url + _endpoint, {
            method: 'GET',
            headers: new Headers({
                'Accept': 'application/json'
            })
        })
        .then(this._handleError)
        .catch( error => { throw new Error(error) });
    },
  
    /**
     * Post abstraction.
     * @return {Promise}
     */
    post(_endpoint, _body) {
        return window.fetch(this.url + _endpoint, {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: _body,
  
        })
        .then(this._handleError)
        .catch( error => { throw new Error(error) });
    }
  };
  

In this module, we have 2 public methods, get() and post() which both return a Promise. On all places where we need to work with remote data, instead of directly calling the Fetch API via window.fetch(), we use our API module abstraction - API.get() or API.post().

Therefore, the Fetch API is not tightly coupled with our code.

Let's say down the road we read Zell Liew's comprehensive summary of using Fetch and we realize that our error handling is not really advanced, like it could be. We want to check the content type before we process with our logic any further. No problem. We modify only our APP module, the public methods API.get() and API.post() we use everywhere else works just fine.

const API = {
    /* ...  */

    /**
     * Check whether the content type is correct before you process it further.
     */
    _handleContentType(_response) {
        const contentType = _response.headers.get('content-type');

        if (contentType && contentType.includes('application/json')) {
            return _response.json();
        }

        return Promise.reject('Oops, we haven\'t got JSON!');
    },

    get(_endpoint) {
        return window.fetch(this.url + _endpoint, {
            method: 'GET',
            headers: new Headers({
                'Accept': 'application/json'
            })
        })
        .then(this._handleError)
        .then(this._handleContentType)
        .catch( error => { throw new Error(error) })
    },

    post(_endpoint, _body) {
        return window.fetch(this.url + _endpoint, {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: _body
        })
        .then(this._handleError)
        .then(this._handleContentType)
        .catch( error => { throw new Error(error) })
    }
};

Let's say we decide to switch to zlFetch, the library which Zell introduces that abstracts away the handling of the response (so you can skip ahead to and handle both your data and errors without worrying about the response). As long as our public methods return a Promise, no problem:

import zlFetch from 'zl-fetch';

const API = {
    /* ...  */

    /**
     * Get abstraction.
     * @return {Promise}
     */
    get(_endpoint) {
        return zlFetch(this.url + _endpoint, {
            method: 'GET'
        })
        .catch( error => { throw new Error(error) })
    },

    /**
     * Post abstraction.
     * @return {Promise}
     */
    post(_endpoint, _body) {
        return zlFetch(this.url + _endpoint, {
            method: 'post',
            body: _body
        })
        .catch( error => { throw new Error(error) });
    }
};

Let's say down the road due to whatever reason we decide to switch to jQuery Ajax for working with remote data. Not a huge deal once again, as long as our public methods return a Promise. The jqXHR objects returned by $.ajax() as of jQuery 1.5 implement the Promise interface, giving them all the properties, methods, and behavior of a Promise.

const API = {
    /* ...  */

    /**
     * Get abstraction.
     * @return {Promise}
     */
    get(_endpoint) {
        return $.ajax({
            method: 'GET',
            url: this.url + _endpoint
        });
    },

    /**
     * Post abstraction.
     * @return {Promise}
     */
    post(_endpoint, _body) {
        return $.ajax({
            method: 'POST',
            url: this.url + _endpoint,
            data: _body
        });
    }
};

But even if jQuery's $.ajax() didn't return a Promise, you can always wrap anything in a new Promise(). All good. Maintainability++!

Now let's abstract away the receiving and storing of the data locally.

Data Repository

Let's assume we need to take the current weather. API returns us the temperature, feels-like, wind speed (m/s), pressure (hPa) and humidity (%). A common pattern, in order for the JSON response to be as slim as possible, attributes are compressed up to the first letter. So here's what we receive from the server:

{
    "t": 30,
    "f": 32,
    "w": 6.7,
    "p": 1012,
    "h": 38
}

We could go ahead and use API.get('weather').t and API.get('weather').w wherever we need it, but that doesn't look semantically awesome. I'm not a fan of the one-letter-not-much-context naming.

Additionally, let's say we don't use the humidity (h) and the feels like temperature (f) anywhere. We don't need them. Actually, the server might return us a lot of other information, but we might want to use only a couple of parameters only. Not restricting what our weather module actually needs (stores) could grow to a big overhead.

Enter repository-ish pattern abstraction!

import API from './api.js'; // Import it into your code however you like

const WeatherRepository = {
    _normalizeData(currentWeather) {
        // Take only what our app needs and nothing more.
        const { t, w, p } = currentWeather;

        return {
            temperature: t,
            windspeed: w,
            pressure: p
        };
    },

    /**
     * Get current weather.
     * @return {Promise}
     */
    get(){
        return API.get('/weather')
            .then(this._normalizeData);
    }
}

Now throughout our codebase use WeatherRepository.get() and access meaningful attributes like .temperature and .windspeed. Better!

Additionally, via the _normalizeData() we expose only parameters we need.

There is one more big benefit. Imagine we need to wire-up our app with another weather API. Surprise, surprise, this one's response attributes names are different:

{
    "temp": 30,
    "feels": 32,
    "wind": 6.7,
    "press": 1012,
    "hum": 38
}

No worries! Having our WeatherRepository abstraction all we need to tweak is the _normalizeData() method! Not a single other module (or file).

const WeatherRepository = {
    _normalizeData(currentWeather) {
        // Take only what our app needs and nothing more.
        const { temp, wind, press } = currentWeather;

        return {
            temperature: temp,
            windspeed: wind,
            pressure: press
        };
    },

    /* ...  */
};

The attribute names of the API response object are not tightly coupled with our codebase. Maintainability++!

Down the road, say we want to display the cached weather info if the currently fetched data is not older than 15 minutes. So, we choose to use localStorage to store the weather info, instead of doing an actual network request and calling the API each time WeatherRepository.get() is referenced.

As long as WeatherRepository.get() returns a Promise, we don't need to change the implementation in any other module. All other modules which want to access the current weather don't (and shouldn't) care how the data is retrieved - if it comes from the local storage, from an API request, via Fetch API or via jQuery's $.ajax(). That's irrelevant. They only care to receive it in the "agreed" format they implemented - a Promise which wraps the actual weather data.

So, we introduce two "private" methods _isDataUpToDate() - to check if our data is older than 15 minutes or not and _storeData() to simply store out data in the browser storage.

const WeatherRepository = {
    /* ...  */

    /**
     * Checks weather the data is up to date or not.
     * @return {Boolean}
     */
    _isDataUpToDate(_localStore) {
        const isDataMissing =
            _localStore === null || Object.keys(_localStore.data).length === 0;

        if (isDataMissing) {
            return false;
        }

        const { lastFetched } = _localStore;
        const outOfDateAfter = 15 * 1000; // 15 minutes

        const isDataUpToDate =
            (new Date().valueOf() - lastFetched) < outOfDateAfter;

        return isDataUpToDate;
    },

    _storeData(_weather) {
        window.localStorage.setItem('weather', JSON.stringify({
            lastFetched: new Date().valueOf(),
            data: _weather
        }));
    },

    /**
     * Get current weather.
     * @return {Promise}
     */
    get(){
        const localData = JSON.parse( window.localStorage.getItem('weather') );

        if (this._isDataUpToDate(localData)) {
            return new Promise(_resolve => _resolve(localData));
        }

        return API.get('/weather')
            .then(this._normalizeData)
            .then(this._storeData);
    }
};

Finally, we tweak the get() method: in case the weather data is up to date, we wrap it in a Promise and we return it. Otherwise - we issue an API call. Awesome!

There could be other use-cases, but I hope you got the idea. If a change requires you to tweak only one module - that's excellent! You designed the implementation in a maintainable way!

If you decide to use this repository-ish pattern, you might notice that it leads to some code and logic duplication, because all data repositories (entities) you define in your project will probably have methods like _isDataUpToDate(), _normalizeData(), _storeData() and so on...

Since I use it heavily in my projects, I decided to create a library around this pattern that does exactly what I described in this article, and more!

Introducing SuperRepo

SuperRepo is a library that helps you implement best practices for working with and storing data on the client-side.

/**
 * 1. Define where you want to store the data,
 *    in this example, in the LocalStorage.
 *
 * 2. Then - define a name of your data repository,
 *    it's used for the LocalStorage key.
 *
 * 3. Define when the data will get out of date.
 *
 * 4. Finally, define your data model, set custom attribute name
 *    for each response item, like we did above with `_normalizeData()`.
 *    In the example, server returns the params 't', 'w', 'p',
 *    we map them to 'temperature', 'windspeed', and 'pressure' instead.
 */
const WeatherRepository = new SuperRepo({
  storage: 'LOCAL_STORAGE',                // [1]
  name: 'weather',                         // [2]
  outOfDateAfter: 5 * 60 * 1000, // 5 min  // [3]
  request: () => API.get('weather'),       // Function that returns a Promise
  dataModel: {                             // [4]
      temperature: 't',
      windspeed: 'w',
      pressure: 'p'
  }
});

/**
 * From here on, you can use the `.getData()` method to access your data.
 * It will first check if out data outdated (based on the `outOfDateAfter`).
 * If so - it will do a server request to get fresh data,
 * otherwise - it will get it from the cache (Local Storage).
 */
WeatherRepository.getData().then( data => {
    // Do something awesome.
    console.log(`It is ${data.temperature} degrees`);
});

The library does the same things we implemented before:

• Gets data from the server (if it's missing or out of date on our side) or otherwise - gets it from the cache.
• Just like we did with _normalizeData(), the dataModel option applies a mapping to our rough data. This means:
  • Throughout our codebase, we will access meaningful and semantic attributes like
  • .temperature and .windspeed instead of .t and .s.
  • Expose only parameters you need and simply don't include any others.
  • If the response attributes names change (or you need to wire-up another API with different response structure), you only need to tweak here - in only 1 place of your codebase.

Plus, a few additional improvements:

• Performance: if WeatherRepository.getData() is called multiple times from different parts of our app, only 1 server request is triggered.
• Scalability:
  • You can store the data in the localStorage, in the browser storage (if you're building a browser extension), or in a local variable (if you don't want to store data across browser sessions). See the options for the storage setting.
  • You can initiate an automatic data sync with WeatherRepository.initSyncer(). This will initiate a setInterval, which will countdown to the point when the data is out of date (based on the outOfDateAfter value) and will trigger a server request to get fresh data. Sweet.

To use SuperRepo, install (or simply download) it with NPM or Bower:

npm install --save super-repo

Then, import it into your code via one of the 3 methods available:

• Static HTML:

  <script src="/node_modules/super-repo/src/index.js"></script>
  

• Using ES6 Imports:

  // If transpiler is configured (Traceur Compiler, Babel, Rollup, Webpack)
  import SuperRepo from 'super-repo';
  

• … or using CommonJS Imports

  // If module loader is configured (RequireJS, Browserify, Neuter)
  const SuperRepo = require('super-repo');
  

And finally, define your SuperRepositories :)

For advanced usage, read the documentation I wrote. Examples included!

Summary

The abstractions I described above could be one fundamental part of the architecture and software design of your app. As your experience grows, try to think about and apply similar concepts not only when working with remote data, but in other cases where they make sense, too.

When implementing a feature, always try to discuss change resilience, maintainability, and scalability with your team. Future you will thank you for that!

---

The Importance Of JavaScript Abstractions When Working With Remote Data is a post from CSS-Tricks

from CSS-Tricks http://ift.tt/2xtMCML
via IFTTT

10 Things that DO NOT (Directly) Affect Your Google Rankings - Whiteboard Friday

Posted by randfish

What do the age of your site, your headline H1/H2 preference, bounce rate, and shared hosting all have in common? You might've gotten a hint from the title: not a single one of them directly affects your Google rankings. In this rather comforting Whiteboard Friday, Rand lists out ten factors commonly thought to influence your rankings that Google simply doesn't care about.

Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we're going to chat about things that do not affect your Google rankings.

So it turns out lots of people have this idea that anything and everything that you do with your website or on the web could have an impact. Well, some things have an indirect impact and maybe even a few of these do. I'll talk through those. But tons and tons of things that you do don't directly affect your Google rankings. So I'll try and walk through some of these that I've heard or seen questions about, especially in the recent past.

1. The age of your website.

First one, longstanding debate: the age of your website. Does Google care if you registered your site in 1998 or 2008 or 2016? No, they don't care at all. They only care the degree to which your content actually helps people and that you have links and authority signals and those kinds of things. Granted, it is true there's correlation going in this direction. If you started a site in 1998 and it's still going strong today, chances are good that you've built up lots of links and authority and equity and all these kinds of signals that Google does care about.

But maybe you've just had a very successful first two years, and you only registered your site in 2015, and you've built up all those same signals. Google is actually probably going to reward that site even more, because it's built up the same authority and influence in a very small period of time versus a much longer one.

2. Whether you do or don't use Google apps and services.

So people worry that, "Oh, wait a minute. Can't Google sort of monitor what's going on with my Google Analytics account and see all my data there and AdSense? What if they can look inside Gmail or Google Docs?"

Google, first off, the engineers who work on these products and the engineers who work on search, most of them would quit right that day if they discovered that Google was peering into your Gmail account to discover that you had been buying shady links or that you didn't look as authoritative as you really were on the web or these kinds of things. So don't fear the use of these or the decision not to use them will hurt or harm your rankings in Google web search in any way. It won't.

3. Likes, shares, plus-ones, tweet counts of your web pages.

So you have a Facebook counter on there, and it shows that you have 17,000 shares on that page. Wow, that's a lot of shares. Does Google care? No, they don't care at all. In fact, they're not even looking at that or using it. But what if it turns out that many of those people who shared it on Facebook also did other activities that resulted in lots of browser activity and search activity, click-through activity, increased branding, lower pogo-sticking rates, brand preference for you in the search results, and links? Well, Google does care about a lot of those things. So indirectly, this can have an impact. Directly, no. Should you buy 10,000 Facebook shares? No, you should not.

4. What about raw bounce rate or time on site?

Well, this is sort of an interesting one. Let's say you have a time on site of two minutes, and you look at your industry averages, your benchmarks, maybe via Google Analytics if you've opted in to sharing there, and you see that your industry benchmarks are actually lower than average. Is that going to hurt you in Google web search? Not necessarily. It could be the case that those visitors are coming from elsewhere. It could be the case that you are actually serving up a faster-loading site and you're getting people to the information that they need more quickly, and so their time on site is slightly lower or maybe even their bounce rate is higher.

But so long as pogo-sticking type of activity, people bouncing back to the search results and choosing a different result because you didn't actually answer their query, so long as that remains fine, you're not in trouble here. So raw bounce rate, raw time on site, I wouldn't worry too much about that.

5. The tech under your site's hood.

Are you using certain JavaScript libraries like Node or React, one is Facebook, one is Google. If you use Facebook's, does Google give you a hard time about it? No. Facebook might, due to patent issues, but anyway we won't worry about that. .NET or what if you're coding up things in raw HTML still? Just fine. It doesn't matter. If Google can crawl each of these URLs and see the unique content on there and the content that Google sees and the content visitors see is the same, they don't care what's being used under the hood to deliver that to the browser.

6. Having or not having a knowledge panel on the right-hand side of the search results.

Sometimes you get that knowledge panel, and it shows around the web and some information sometimes from Wikipedia. What about site links, where you search for your brand name and you get branded site links? The first few sets of results are all from your own website, and they're sort of indented. Does that impact your rankings? No, it does not. It doesn't impact your rankings for any other search query anyway.

It could be that showing up here and it probably is that showing up here means you're going to get a lot more of these clicks, a higher share of those clicks, and it's a good thing. But does this impact your rankings for some other totally unbranded query to your site? No, it doesn't at all. I wouldn't stress too much. Over time, sites tend to build up site links and knowledge panels as their brands become bigger and as they become better known and as they get more coverage around the web and online and offline. So this is not something to stress about.

7. What about using shared hosting or some of the inexpensive hosting options out there?

Well, directly, this is not going to affect you unless it hurts load speed or up time. If it doesn't hurt either of those things and they're just as good as they were before or as they would be if you were paying more or using solo hosting, you're just fine. Don't worry about it.

8. Use of defaults that Google already assumes.

So when Google crawls a site, when they come to a site, if you don't have a robots.txt file, or you have a robots.txt file but it doesn't include any exclusions, any disallows, or they reach a page and it has no meta robots tag, they're just going to assume that they get to crawl everything and that they should follow all the links.

Using things like the meta robots "index, follow" or using, on an individual link, a rel=follow inside the href tag, or in your robots.txt file specifying that Google can crawl everything, doesn't boost anything. They just assume all those things by default. Using them in these places, saying yes, you can do the default thing, doesn't give you any special benefit. It doesn't hurt you, but it gives you no benefit. Google just doesn't care.

9. Characters that you use as separators in your title element.

So the page title element sits in the header of a document, and it could be something like your brand name and then a separator and some words and phrases after it, or the other way around, words and phrases, separator, the brand name. Does it matter if that separator is the pipe bar or a hyphen or a colon or any other special character that you would like to use? No, Google does not care. You don't need to worry about it. This is a personal preference issue.

Now, maybe you've found that one of these characters has a slightly better click-through rate and preference than another one. If you've found that, great. We have not seen one broadly on the web. Some people will say they particularly like the pipe over the hyphen. I don't think it matters too much. I think it's up to you.

10. What about using headlines and the H1, H2, H3 tags?

Well, I've heard this said: If you put your headline inside an H2 rather than an H1, Google will consider it a little less important. No, that is definitely not true. In fact, I'm not even sure the degree to which Google cares at all whether you use H1s or H2s or H3s, or whether they just look at the content and they say, "Well, this one is big and at the top and bold. That must be the headline, and that's how we're going to treat it. This one is lower down and smaller. We're going to say that's probably a sub-header."

Whether you use an H5 or an H2 or an H3, that is your CSS on your site and up to you and your designers. It is still best practices in HTML to make sure that the headline, the biggest one is the H1. I would do that for design purposes and for having nice clean HTML and CSS, but I wouldn't stress about it from Google's perspective. If your designers tell you, "Hey, we can't get that headline in H1. We've got to use the H2 because of how our style sheets are formatted." Fine. No big deal. Don't stress.

Normally on Whiteboard Friday, we would end right here. But today, I'd like to ask. These 10 are only the tip of the iceberg. So if you have others that you've seen people say, "Oh, wait a minute, is this a Google ranking factor?" and you think to yourself, "Ah, jeez, no, that's not a ranking factor," go ahead and leave them in the comments. We'd love to see them there and chat through and list all the different non-Google ranking factors.

Thanks, everyone. See you again next week for another edition of Whiteboard Friday. Take care.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

from The Moz Blog http://ift.tt/2xz3Bh4
via IFTTT

Creating a Static API from a Repository

When I first started building websites, the proposition was quite basic: take content, which may or may not be stored in some form of database, and deliver it to people's browsers as HTML pages. Over the years, countless products used that simple model to offer all-in-one solutions for content management and delivery on the web.

Fast-forward a decade or so and developers are presented with a very different reality. With such a vast landscape of devices consuming digital content, it's now imperative to consider how content can be delivered not only to web browsers, but also to native mobile applications, IoT devices, and other mediums yet to come.

Even within the realms of the web browser, things have also changed: client-side applications are becoming more and more ubiquitous, with challenges to content delivery that didn't exist in traditional server-rendered pages.

The answer to these challenges almost invariably involves creating an API — a way of exposing data in such a way that it can be requested and manipulated by virtually any type of system, regardless of its underlying technology stack. Content represented in a universal format like JSON is fairly easy to pass around, from a mobile app to a server, from the server to a client-side application and pretty much anything else.

Embracing this API paradigm comes with its own set of challenges. Designing, building and deploying an API is not exactly straightforward, and can actually be a daunting task to less experienced developers or to front-enders that simply want to learn how to consume an API from their React/Angular/Vue/Etc applications without getting their hands dirty with database engines, authentication or data backups.

Back to Basics

I love the simplicity of static sites and I particularly like this new era of static site generators. The idea of a website using a group of flat files as a data store is also very appealing to me, which using something like GitHub means the possibility of having a data set available as a public repository on a platform that allows anyone to easily contribute, with pull requests and issues being excellent tools for moderation and discussion.

Imagine having a site where people find a typo in an article and submit a pull request with the correction, or accepting submissions for new content with an open forum for discussion, where the community itself can filter and validate what ultimately gets published. To me, this is quite powerful.

I started toying with the idea of applying these principles to the process of building an API instead of a website — if programs like Jekyll or Hugo take a bunch of flat files and create HTML pages from them, could we build something to turn them into an API instead?

Static Data Stores

Let me show you two examples that I came across recently of GitHub repositories used as data stores, along with some thoughts on how they're structured.

The first example is the ESLint website, where every single ESLint rule is listed along with its options and associated examples of correct and incorrect code. Information for each rule is stored in a Markdown file annotated with a YAML front matter section. Storing the content in this human-friendly format makes it easy for people to author and maintain, but not very simple for other applications to consume programmatically.

The second example of a static data store is MDN's browser-compat-data, a compendium of browser compatibility information for CSS, JavaScript and other technologies. Data is stored as JSON files, which conversely to the ESLint case, are a breeze to consume programmatically but a pain for people to edit, as JSON is very strict and human errors can easily lead to malformed files.

There are also some limitations stemming from the way data is grouped together. ESLint has a file per rule, so there's no way to, say, get a list of all the rules specific to ES6, unless they chuck them all into the same file, which would be highly impractical. The same applies to the structure used by MDN.

A static site generator solves these two problems for normal websites — they take human-friendly files, like Markdown, and transform them into something tailored for other systems to consume, typically HTML. They also provide ways, through their template engines, to take the original files and group their rendered output in any way imaginable.

Similarly, the same concept applied to APIs — a static API generator? — would need to do the same, allowing developers to keep data in smaller files, using a format they're comfortable with for an easy editing process, and then process them in such a way that multiple endpoints with various levels of granularity can be created, transformed into a format like JSON.

Building a Static API Generator

Imagine an API with information about movies. Each title should have information about the runtime, budget, revenue, and popularity, and entries should be grouped by language, genre, and release year.

To represent this dataset as flat files, we could store each movie and its attributes as a text, using YAML or any other data serialization language.

budget: 170000000
website: http://ift.tt/1nPjEaW
tmdbID: 118340
imdbID: tt2015381
popularity: 50.578093
revenue: 773328629
runtime: 121
tagline: All heroes start somewhere.
title: Guardians of the Galaxy

To group movies, we can store the files within language, genre and release year sub-directories, as shown below.

input/
├── english
│   ├── action
│   │   ├── 2014
│   │   │   └── guardians-of-the-galaxy.yaml
│   │   ├── 2015
│   │   │   ├── jurassic-world.yaml
│   │   │   └── mad-max-fury-road.yaml
│   │   ├── 2016
│   │   │   ├── deadpool.yaml
│   │   │   └── the-great-wall.yaml
│   │   └── 2017
│   │       ├── ghost-in-the-shell.yaml
│   │       ├── guardians-of-the-galaxy-vol-2.yaml
│   │       ├── king-arthur-legend-of-the-sword.yaml
│   │       ├── logan.yaml
│   │       └── the-fate-of-the-furious.yaml
│   └── horror
│       ├── 2016
│       │   └── split.yaml
│       └── 2017
│           ├── alien-covenant.yaml
│           └── get-out.yaml
└── portuguese
    └── action
        └── 2016
            └── tropa-de-elite.yaml

Without writing a line of code, we can get something that is kind of an API (although not a very useful one) by simply serving the `input/` directory above using a web server. To get information about a movie, say, Guardians of the Galaxy, consumers would hit:

http://localhost/english/action/2014/guardians-of-the-galaxy.yaml

and get the contents of the YAML file.

Using this very crude concept as a starting point, we can build a tool — a static API generator — to process the data files in such a way that their output resembles the behavior and functionality of a typical API layer.

Format translation

The first issue with the solution above is that the format chosen to author the data files might not necessarily be the best format for the output. A human-friendly serialization format like YAML or TOML should make the authoring process easier and less error-prone, but the API consumers will probably expect something like XML or JSON.

Our static API generator can easily solve this by visiting each data file and transforming its contents to JSON, saving the result to a new file with the exact same path as the source, except for the parent directory (e.g. `output/` instead of `input/`), leaving the original untouched.

This results on a 1-to-1 mapping between source and output files. If we now served the `output/` directory, consumers could get data for Guardians of the Galaxy in JSON by hitting:

http://localhost/english/action/2014/guardians-of-the-galaxy.json

whilst still allowing editors to author files using YAML or other.

{
  "budget": 170000000,
  "website": "http://ift.tt/1nPjEaW",
  "tmdbID": 118340,
  "imdbID": "tt2015381",
  "popularity": 50.578093,
  "revenue": 773328629,
  "runtime": 121,
  "tagline": "All heroes start somewhere.",
  "title": "Guardians of the Galaxy"
}

Aggregating data

With consumers now able to consume entries in the best-suited format, let's look at creating endpoints where data from multiple entries are grouped together. For example, imagine an endpoint that lists all movies in a particular language and of a given genre.

The static API generator can generate this by visiting all subdirectories on the level being used to aggregate entries, and recursively saving their sub-trees to files placed at the root of said subdirectories. This would generate endpoints like:

http://localhost/english/action.json

which would allow consumers to list all action movies in English, or

http://localhost/english.json

to get all English movies.

{  
   "results": [  
      {  
         "budget": 150000000,
         "website": "http://ift.tt/2am8UEO",
         "tmdbID": 311324,
         "imdbID": "tt2034800",
         "popularity": 21.429666,
         "revenue": 330642775,
         "runtime": 103,
         "tagline": "1700 years to build. 5500 miles long. What were they trying to keep out?",
         "title": "The Great Wall"
      },
      {  
         "budget": 58000000,
         "website": "http://ift.tt/19ZQQxf",
         "tmdbID": 293660,
         "imdbID": "tt1431045",
         "popularity": 23.993667,
         "revenue": 783112979,
         "runtime": 108,
         "tagline": "Witness the beginning of a happy ending",
         "title": "Deadpool"
      }
   ]
}

To make things more interesting, we can also make it capable of generating an endpoint that aggregates entries from multiple diverging paths, like all movies released in a particular year. At first, it may seem like just another variation of the examples shown above, but it's not. The files corresponding to the movies released in any given year may be located at an indeterminate number of directories — for example, the movies from 2016 are located at `input/english/action/2016`, `input/english/horror/2016` and `input/portuguese/action/2016`.

We can make this possible by creating a snapshot of the data tree and manipulating it as necessary, changing the root of the tree depending on the aggregator level chosen, allowing us to have endpoints like http://localhost/2016.json.

Pagination

Just like with traditional APIs, it's important to have some control over the number of entries added to an endpoint — as our movie data grows, an endpoint listing all English movies would probably have thousands of entries, making the payload extremely large and consequently slow and expensive to transmit.

To fix that, we can define the maximum number of entries an endpoint can have, and every time the static API generator is about to write entries to a file, it divides them into batches and saves them to multiple files. If there are too many action movies in English to fit in:

http://localhost/english/action.json

we'd have

http://localhost/english/action-2.json

and so on.

For easier navigation, we can add a metadata block informing consumers of the total number of entries and pages, as well as the URL of the previous and next pages when applicable.

{  
   "results": [  
      {  
         "budget": 150000000,
         "website": "http://ift.tt/2am8UEO",
         "tmdbID": 311324,
         "imdbID": "tt2034800",
         "popularity": 21.429666,
         "revenue": 330642775,
         "runtime": 103,
         "tagline": "1700 years to build. 5500 miles long. What were they trying to keep out?",
         "title": "The Great Wall"
      },
      {  
         "budget": 58000000,
         "website": "http://ift.tt/19ZQQxf",
         "tmdbID": 293660,
         "imdbID": "tt1431045",
         "popularity": 23.993667,
         "revenue": 783112979,
         "runtime": 108,
         "tagline": "Witness the beginning of a happy ending",
         "title": "Deadpool"
      }
   ],
   "metadata": {  
      "itemsPerPage": 2,
      "pages": 3,
      "totalItems": 6,
      "nextPage": "/english/action-3.json",
      "previousPage": "/english/action.json"
   }
}

Sorting

It's useful to be able to sort entries by any of their properties, like sorting movies by popularity in descending order. This is a trivial operation that takes place at the point of aggregating entries.

Putting it all together

Having done all the specification, it was time to build the actual static API generator app. I decided to use Node.js and to publish it as an npm module so that anyone can take their data and get an API off the ground effortlessly. I called the module static-api-generator (original, right?).

To get started, create a new folder and place your data structure in a sub-directory (e.g. `input/` from earlier). Then initialize a blank project and install the dependencies.

npm init -y
npm install static-api-generator --save

The next step is to load the generator module and create an API. Start a blank file called `server.js` and add the following.

const API = require('static-api-generator')
const moviesApi = new API({
  blueprint: 'source/:language/:genre/:year/:movie',
  outputPath: 'output'
})

In the example above we start by defining the API blueprint, which is essentially naming the various levels so that the generator knows whether a directory represents a language or a genre just by looking at its depth. We also specify the directory where the generated files will be written to.

Next, we can start creating endpoints. For something basic, we can generate an endpoint for each movie. The following will give us endpoints like /english/action/2016/deadpool.json.

moviesApi.generate({
  endpoints: ['movie']
})

We can aggregate data at any level. For example, we can generate additional endpoints for genres, like /english/action.json.

moviesApi.generate({
  endpoints: ['genre', 'movie']
})

To aggregate entries from multiple diverging paths of the same parent, like all action movies regardless of their language, we can specify a new root for the data tree. This will give us endpoints like /action.json.

moviesApi.generate({
  endpoints: ['genre', 'movie'],
  root: 'genre'
})

By default, an endpoint for a given level will include information about all its sub-levels — for example, an endpoint for a genre will include information about languages, years and movies. But we can change that behavior and specify which levels to include and which ones to bypass.

The following will generate endpoints for genres with information about languages and movies, bypassing years altogether.

moviesApi.generate({
  endpoints: ['genre'],
  levels: ['language', 'movie'],
  root: 'genre'
})

Finally, type npm start to generate the API and watch the files being written to the output directory. Your new API is ready to serve - enjoy!

Deployment

At this point, this API consists of a bunch of flat files on a local disk. How do we get it live? And how do we make the generation process described above part of the content management flow? Surely we can't ask editors to manually run this tool every time they want to make a change to the dataset.

GitHub Pages + Travis CI

If you're using a GitHub repository to host the data files, then GitHub Pages is a perfect contender to serve them. It works by taking all the files committed to a certain branch and making them accessible on a public URL, so if you take the API generated above and push the files to a gh-pages branch, you can access your API on http://ift.tt/2hkkcvl.

We can automate the process with a CI tool, like Travis. It can listen for changes on the branch where the source files will be kept (e.g. master), run the generator script and push the new set of files to gh-pages. This means that the API will automatically pick up any change to the dataset within a matter of seconds – not bad for a static API!

After signing up to Travis and connecting the repository, go to the Settings panel and scroll down to Environment Variables. Create a new variable called GITHUB_TOKEN and insert a GitHub Personal Access Token with write access to the repository – don't worry, the token will be safe.

Finally, create a file named `.travis.yml` on the root of the repository with the following.

language: node_js

node_js:
  - "7"

script: npm start

deploy:
  provider: pages
  skip_cleanup: true
  github_token: $GITHUB_TOKEN
  on:
    branch: master
  local_dir: "output"

And that's it. To see if it works, commit a new file to the master branch and watch Travis build and publish your API. Ah, GitHub Pages has full support for CORS, so consuming the API from a front-end application using Ajax requests will work like a breeze.

You can check out the demo repository for my Movies API and see some of the endpoints in action:

• Movie endpoint (Deadpool)
• List of genres with languages and years
• List of languages and years by genre (Action)
• Full list of languages with genres, years and movies

Going full circle with Staticman

Perhaps the most blatant consequence of using a static API is that it's inherently read-only – we can't simply set up a POST endpoint to accept data for new movies if there's no logic on the server to process it. If this is a strong requirement for your API, that's a sign that a static approach probably isn't the best choice for your project, much in the same way that choosing Jekyll or Hugo for a site with high levels of user-generated content is probably not ideal.

But if you just need some basic form of accepting user data, or you're feeling wild and want to go full throttle on this static API adventure, there's something for you. Last year, I created a project called Staticman, which tries to solve the exact problem of adding user-generated content to static sites.

It consists of a server that receives POST requests, submitted from a plain form or sent as a JSON payload via Ajax, and pushes data as flat files to a GitHub repository. For every submission, a pull request will be created for your approval (or the files will be committed directly if you disable moderation).

You can configure the fields it accepts, add validation, spam protection and also choose the format of the generated files, like JSON or YAML.

This is perfect for our static API setup, as it allows us to create a user-facing form or a basic CMS interface where new genres or movies can be added. When a form is submitted with a new entry, we'll have:

• Staticman receives the data, writes it to a file and creates a pull request
• As the pull request is merged, the branch with the source files (master) will be updated
• Travis detects the update and triggers a new build of the API
• The updated files will be pushed to the public branch (gh-pages)
• The live API now reflects the submitted entry.

Parting thoughts

To be clear, this article does not attempt to revolutionize the way production APIs are built. More than anything, it takes the existing and ever-popular concept of statically-generated sites and translates them to the context of APIs, hopefully keeping the simplicity and robustness associated with the paradigm.

In times where APIs are such fundamental pieces of any modern digital product, I'm hoping this tool can democratize the process of designing, building and deploying them, and eliminate the entry barrier for less experienced developers.

The concept could be extended even further, introducing concepts like custom generated fields, which are automatically populated by the generator based on user-defined logic that takes into account not only the entry being created, but also the dataset as a whole – for example, imagine a rank field for movies where a numeric value is computed by comparing the popularity value of an entry against the global average.

If you decide to use this approach and have any feedback/issues to report, or even better, if you actually build something with it, I'd love to hear from you!

References

• static-api-generator on GitHub
• movies-api on GitHub
• Staticman on GitHub

---

Creating a Static API from a Repository is a post from CSS-Tricks

from CSS-Tricks http://ift.tt/2hkAojz
via IFTTT

​No Joke…Download Anything You Want on Storyblocks