Hi! It’s been a long time since I’ve posted here, but I thought anyone that liked my old posts here might appreciate this:

I’m excited to announce that I’ll be streaming web development coding sessions on Twitch every Sunday morning, starting a few weeks ago!

I am covering a variety of topics, from beginner-friendly tutorials to more advanced concepts. Whether you’re just starting out with web development or you’re a seasoned pro, I hope you’ll join me for some fun and learning.

You can find my Twitch channel at twitch.tv/saebyn. I’ll be streaming live every Sunday morning at 8:30am Pacific (11:30am Eastern, 3:30 PM UTC). I hope to see you there!

What will I be covering?

I’ll be covering a variety of topics during my streams, including:

• HTML and CSS
• Typscript
• React
• Node.js
• Redux
• Prisma
• TRPC
• and more!

Who is this for?

My streams are for everyone, from beginners to experienced developers. I’ll be covering a variety of topics, so there’s something for everyone.

How can I join in?

To join in on the fun, simply head over to twitch.tv/saebyn on Sunday mornings at 8:30am Pacific (11:30am Eastern, 3:30 PM UTC). I’ll be streaming live and answering questions in the chat.

I hope to see you there!

More info

I will be streaming games on weekday evenings in addition to the coding streams on Sunday. I’ve also been uploading my streams to youtube at youtube.com/@saebynVODs if you want to see past streams there instead.

tl;dr, clone https://github.com/saebyn/my-react-app

Introduction

For this setup, I have node v6.9.1 installed. This uses create-react-app to create the base project, and then walks through various things I usually want for a React project, like my preferred editor configuration (for VS Code) for the project, Wallaby.JS, Prettier, and Storybook.

npx create-react-app my-react-app
cd my-react-app
npm i

Validate that the app starts

npm start
open http://localhost:3000 # the app should do this for you

You should see a “Welcome to React” message, logo, etc in your web browser.

Configure Wallaby.js

Grab the Wallaby.js configuration file and save it as “wallaby.js” at the top level of the project.

Editor configuration

1. At the top level of the project, add the following to a “.editorconfig” file:

    [*.js,*.json]
    indent_style = space
    indent_size = 2
   

2. Add extensions to the extensions.json for the workspace (e.g. in .vscode/extensions.json)

    {
      // See http://go.microsoft.com/fwlink/?LinkId=827846
      // for the documentation about the extensions.json format
      "recommendations": [
        // Extension identifier format: ${publisher}.${name}. Example: vscode.csharp
        "EditorConfig.editorconfig",
        "dbaeumer.vscode-eslint",
        "esbenp.prettier-vscode"
      ]
    }
   

3. Then go install those if you don’t already have them.

4. Make vscode apply formatting for this workspace, add this to .vscode/settings.json

    {
      "editor.formatOnSave": true,
      "editor.formatOnType": true
    }
   

Setup prettier

Install prettier and an eslint configuration:

npm i --save-dev --save-exact prettier
npm i --save-dev eslint-config-prettier

Add a .eslintrc.json file to the top level:

{
  "extends": ["prettier", "react-app"]
}

Setup storybook

1. Install it

    npm i --save-dev @storybook/react
   

2. Add an entry to package.json scripts:

    {
      "scripts": {
        ...,
        "storybook": "start-storybook -p 9001 -c .storybook"
      }
    }
   

3. Set up an initial config in .storybook/config.js

    import { configure } from '@storybook/react';
    function loadStories() {
        require('../stories/index.js');
        // You can require as many stories as you need.
    }
    configure(loadStories, module);
   

4. Install Storybook add-ons

    npm install --save-dev @storybook/addon-storyshots @storybook/addon-actions  @storybook/addon-a11y @storybook/addon-links @storybook/addon-graphql @storybook/addon-knobs @storybook/addon-notes react-test-renderer
   

5. Create .storybook/addons.js

    import "@storybook/addon-actions/register";
    import "@storybook/addon-a11y/register";
    import "@storybook/addon-links/register";
    import "@storybook/addon-knobs/register";
    import "@storybook/addon-notes/register";
   

6. Setup storyshots for Jest

   See more about Storyshots and more info about Jest’s snapshot testing.

   Create a new test file with the name Storyshots.test.js (I’m putting it in src/)

    import initStoryshots from '@storybook/addon-storyshots';
    initStoryshots({
        storyNameRegex: /^((?!.*?DontTest).)*$/
    });
   

7. Add some stories just to validate that Storybook is working

   Add this to stories/index.js:

   import React from "react";
   import { storiesOf } from "@storybook/react";
   import { action } from "@storybook/addon-actions";
   import { checkA11y } from "@storybook/addon-a11y";
   import { linkTo } from "@storybook/addon-links";
   import { setupGraphiQL } from "@storybook/addon-graphql";
   import { withKnobs, text, boolean, number } from "@storybook/addon-knobs/react";
   import { withNotes } from "@storybook/addon-notes";
   // setup the graphiql helper which can be used with the add method later
   const graphiql = setupGraphiQL({ url: "http://localhost:3100/graphql" });
   storiesOf("button", module)
       .addDecorator(checkA11y)
       .addDecorator(withKnobs)
       .add(
           "with text",
           withNotes(`
   Clicking this button will take you to the next story.
   `)(() => (
       <button onClick={linkTo("button", "with some emoji")}>
           Hello Button
               </button>
           ))
       )
       .add("with some emoji", () => (
           <button onClick={action("clicked")}>😀 😎 👍 💯</button>
       ))
       .add("Inaccessible", () => (
           <button style=>
               Inaccessible button
           </button>
       ))
       .add(
           "get user info DontTest",
           graphiql(`{
               user(id: "1") {
                   name
               }
           }`)
       )
       .add("with a button", () => (
           <button disabled={boolean("Disabled", false)}>
               {text("Label", "Hello Button")}
           </button>
       ))
       .add("as dynamic variables", () => {
           const name = text("Name", "Arunoda Susiripala");
           const age = number("Age", 89);
           const content = `I am ${name} and I'm ${age} years old.`;
           return <div>{content}</div>;
       });
   

Testing the tests

1. Run the tests

    npm test
   

2. If you get an error about “jest-cli”, try deleting node_modules and running “npm install” again. I had to do this.

3. After it runs, you’ll see a new file: src/__snapshots__/Storyshots.test.js.snap

4. Start Storybook

    npm run storybook
    open http://localhost:9001  # It will not open this automatically (but it will reload when stories are changed.)
   

Check out the end result, or try it yourself.

After five months of working at Saltbox in early 2012, funds ran out. I seriously considered taking a gamble with a work-for-equity offer, but I didn’t see at the time how I could make ends meet while doing so. I left.

I liked the team I worked with in those few months, as you could probably tell from my past blog posts. Ali and I kept in touch via IM occasionally - and that’s how the Tin Khan project came about.

By October, Saltbox completed the pivot to working on Wax, their learning record store. The old “SaLTBOX” app shut down. Wax LRS was now the primary product of Saltbox.

A week before Halloween, Ali and I had already talked about the Tin Can API (now the Experience API) and it piqued my interest. I floated the idea of doing a little demo project, since I had some spare time. The idea was: pulling student data from Khan Academy’s API and putting it into an LRS.

He said:

A vague demo that can connect to Wax would be enough to blow their minds. Better yet, if you could get it done by Wednesday… we can get it to DevLearn for you in Vegas! :)

This was Wednesday, the week before DevLearn. So I slapped together a Django (a web framework for Python) app that let you add a list of students and an LRS endpoint, then used Celery (a distributed task queue) to send an email to each student asking them to click a link. That link takes them back into Tin Khan, pushes them through the OAuth process for the Khan Academy API, and then pulls their activities (videos watched, badges earned, and exercises done) into a local database. New activities were discovered and synchronized to the configured LRS via Tin Can statements sent via HTTP.

One week. The final result was something that worked as a demo, but would need some actual architecture for real world use. Pretty much what people mean when they talk about an “MVP” - a minimum viable product.

Unfortunately, the Khan Academy API doesn’t provide a way to request just the information about a user’s newest activities (i.e. what’s changed since the last time we looked). That means that Tin Khan had to fetch all of the activities every time and store them locally, so that it could compare with what had already been sent to the LRS previously and send the difference. That works for a demo, but not for real use.

It now (four years later) looks like the Khan Academy API supports constraining the date range when fetching a user’s watched videos. If I were to build Tin Khan today, that’s where I would start.

Four years ago, other projects took precedence, and this project was shelved. A few days ago, as I was going through a list of all the projects I’ve worked on in the past few years, I thought of this project. I still had the source sitting in an archive on Dropbox, so I decided I’d put it up on GitHub for future reference (or laughs ;).

Saltbox has occupied a large part of my life for the past five years. I’ve passed on other opportunities to focus on this team and the product we were building, and I’ve learned a lot in the process.

Why Saltbox? The people, many of whom I’ve previously praised in Not Thirsty for Cool Projects at Saltbox, are the core of why I keep finding myself back there. They are focused, driven, and dedicated; but also friendly, supportive, and professional.

Saltbox has had many growing pains, as should be expected in a bootstrapped “startup”, including pivots and cash-flow issues. The company went through its major pivot in product and focus back in mid-2012. And while I still think back on that initial application I worked on, back when I wrote “Not Thirsty …”, as something with a lot of potential, it’s nothing in comparison to Wax LRS.

Wax LRS is a “web scale”¹ platform for receiving data about what learners are doing, supporting an open specification that lets anyone define statements² that describe some activity. These statements include a core tuple of who, what, how, and when; along with associated metadata that can include scoring, categorization, and whatever else with support for extensions to the schema.

Built on top of a data platform to receive and store all the learning, Wax LRS provides “RESTful” interfaces to derive useful insights into the why’s of learning: Are your companies salespeople struggling to absorb knowledge about new products? What learning activities are the best indicator of later success in their jobs, and how is that different for different departments or teams?

The scope of what was possible with Wax LRS, in terms of the potential sources of data to drive reports and visualizations, as well as the possibilities for making a large impact in the professional lives of an unbounded number of learners that can be guided by L&D leaders using the knowledge they could gain from our product, really appeals to me. This is the kind of thing that excites me, and makes me want to keep coming back.

I’m an avid fan fiction reader, and one of my on-going projects is Fanonic.net - a fan fiction hosting site that provides helpful features for readers to find the kind of stories they are looking for more easily. If you haven’t heard of the term or come across fan fiction before, here’s the Wikipedia defintition from the fan fiction page:

Fan fiction (alternatively referred to as fanfiction, fanfic, FF, or fic) is a broadly-defined term for fan labor regarding stories about characters or settings written by fans of the original work, rather than by the original creator. Works of fan fiction are rarely commissioned or authorized by the original work’s owner, creator, or publisher; also, they are almost never professionally published. Because of this, many fan fictions written often contain a disclaimer stating that the creator of the work owns none of the characters. Fan fiction, therefore, is defined by being both related to its subject’s canonical fictional universe and simultaneously existing outside the canon of that universe.

Fanonic is free for both readers and authors. It’s main features, which I developed last year as part of its initial launch, are:

• the ability to search within the entire text of stories
• story tagging by users
• an activity stream that lets you see when your favorite authors add new stores or publish new chapters and when friends earn new achievement badges.

What’s Been Added

• Avatars: Your profile automatically uses your Gravitar image if it’s set, and you can also upload an image as your Fanonic avatar.

• Badges: A few new achievement badges have been added

• Favorites: Stories can be favorited and show up on your list of favorites

• Fanfiction.net import: Import your stories from fanfiction.net into your Fanonic story list

• Lots of style improvements

What’s Coming Next

I’m using Haystack to provide story search. Fanonic is hosted on a single server and is using Whoosh to index the site’s story data, but my intention is to switch to another indexer/backend for Haystack to allow for future scalability. Elastic Search looks like the best direction to go right now because it looks easier to set up and administer than Solr and should be easier to scale to multiple search index servers than Whoosh or Xapian.

The site will be getting immediate notifications for each user’s activity stream and author’s story import status updates. I’ve set up a new repository on github for this subproject: django-tsuchi. I’m also looking into adding reader statistics for story authors, but I haven’t decided on specifics at this point. Support for importing stories from other fan fiction repositories will be added gradually as well.

I’m getting the word out about Fanonic, and getting in touch with some fanfic authors about bringing their stories to the site. I’m hoping to work with both fanfic creators and readers to build a better experience for everyone.

Technical Details

Django, nginx, uwsgi, redis, memcached, postgresql, celery

• The new story importer uses Celery to import the content in the in a separate worker process so users can continue using the site while their content is loaded into Fanonic. I’m using redis to hold the queued tasks. Later on, I’m planning on using redis for some future features involving gathering reader statistics as well.

• Django is a web framework written in Python that provides a lot of the basic funcionality that Fanonic builds upon. Django has a large and growing ecosystem of third-party pluggable apps. I’ve published one such app that is going to provide a new feature for the site, which I’m calling django-tsuchi.

• nginx serves static content, such as Javascript and CSS files, and acts as a proxy to uwsgi.

• uwsgi interfaces with the Fanonic Python code to handle incoming requests and return the app’s responses.

• Fanonic’s database is PostgreSQL.

• Fanonic uses memcached to cache content generated by the backend.

Puppet

Puppet is a tool that allows system administrators to define how the servers they administer are configured, which programs are installed, which services should be running, and what user accounts should be present on them. I’m using Puppet to manage both my local development system and the production server that Fanonic runs. Because both systems have identical setups, I don’t have to worry about differences in the server setup breaking the site - what works locally is much more likely to work on the production server.

• I found some helpful Puppet modules on github:

  • example42 has quite a few Puppet modules on github. I’m using the redis module, and my fork of their nginx module. Many of their modules depend on puppi, which I’ve forked to handle some breakage because of the version of Puppet I’m using (2.7.17).

  • I’m using my fork of the memcached module from danakim to set up memcached.

  • I’m using the postgresql module from uggedal to set up PostgreSQL.

• My local machine has a vagrant box running which automatically loads up the Puppet configuration when it boots.

Fabric

• I’m using Fabric to handle code deployment

• Right now I’m using Fabric to push the Puppet manifests to the server and run puppet apply on them, rather than having a puppetmaster.

• I’ve taken the common tasks for this kind of set up into a Python module of Fabric tasks.