Carbide Alpha

Carbide is a new kind of programming environment which (as obligatory in this day and age):

  • Requires no installation or setup
  • Supports Javascript/ES2015
  • Imports modules automatically from NPM or GitHub
  • Saves and loads directly to and from GitHub Gists
But wait, there's more...

...Crazy Stuff

  • Comments live in a Rich Text sidebar #LiterateProgramming
  • Carbide shows results in-between the lines of your code— letting you see data in context
  • The result of any expression can be logged, visualized, and directly manipulated with powerful (and shiny) widgets
  • Carbide doesn't distinguish between input and output, using generalized machine learning techniques to run programs backwards

Carbide is pretty weird (not just because it's still fairly buggy), so we highly recommend at least skimming the Quick Start Guide. But if you're truly an unstoppable force, you can also jump straight to some examples.

Quick Start

At the end of this guide there will be links to example notebooks.

Running Code

Running code in Carbide is as simple as hitting the "Run" button located on the bottom right of any cell. You can also quickly run the current cell with (Cmd-Enter).

Currently Carbide supports the latest version of Javascript, known as ES2015 through built-in support for BabelJS. You can even use ES7's async/await features without wrapping your code in functions.

In addition to running code locally within the editor, you can choose to run code within a separate window (this is nice for building out apps which need access to the DOM). To switch to this mode, click on the language dropdown on the top of the page, and select "Javascript (Separate Window)".

We haven't launched support for other programming languages yet (rest assured it's on our roadmap).

Attaching Probes

In Carbide, you see the results of running your program in "probes" which are displayed in between the lines of your code.

You can inspect any expression by selecting it and hitting Cmd-I.

Rather than littering your code with print statements and breaking up nested expressions, or painstakingly stepping through with a traditional debugger, it’s easy to see an overview of all the values as they flow through a program.

As a convenience, the outermost expression on any line is automatically logged, and a widget is shown for the last expression of the cell.

Visualization Widgets

When you want to see more than the basic summary probes show you, you can click any probe to open a "widget", which appears to the right of your code.

Hovering over a widget expands a tab bar on top, which allows you to switch between different representations of your data. A string like "orange" can be visualized as color, as plain text, or as JSON, and widgets allow you to easily switch between them.

Beyond simply looking at a piece of data, visualization widgets can also incorporate knowledge of your code's abstract syntax tree. This allows us to visualize array[0] as a selector into the array, or “house” == “horse” as an interactive string comparison, rather than the uninformative false.

You can see more examples of these kinds of widgets in the Widget Menagerie section.

Manipulation Widgets

The exact same widgets which were used for visualizing data can also be used for manipulating data. This is an idea which really lies at the heart of what Carbide is— peeling away the one-way mirror found in most data visualization tools which forces you to look and not touch.

For example, we could represent the number "42" as a slider with the bounds set between 0 and 100. As a visualization, this gives me some spatial sense of how large this number is. When I drag the slider, it would make sense for the part of the code which said "42" to be replaced with the new value.

We start to get into trouble when the "42" isn't actually found in the code, and values are computed rather than literal. To figure this out we have to essentially run the program backwards— fixing the outputs, and solving for the inputs. To do this, we use a technique from machine learning, called backpropagation. If you're interested, there's a longer technical explanation of what it is and how it works below.

Multiple Cells

Carbide is a notebook programming environment, which means that the code is stuck into little boxes called cells. Each cell is a little playground where you can build stuff, aided by all the fancy tools for seeing, poking, and prodding.

To insert a cell, click in the area either immediately before or after a cell.

Unlike most other notebook programming systems, you'll find yourself needing to make cells much less frequently in Carbide because you don't need to break up your program into multiple cells to visualize what's going on.

With multiple cells you can try out multiple approaches to a given problem at the same time without necessarily committing to any particular change.

Each cell has a file name, which is shown in the cell toolbar on the bottom of each cell. This filename comes into play when sharing information between cells with the ES6 import/export syntax. 

// cell0.js
export default 420 
// cell1.js
import data from './cell0.js'
data + 2

Saving and Sharing

Currently, all Carbide notebooks are represented as public GitHub Gists.

If you are not logged in, hitting Cmd-S or pressing the "Save" button creates a new Gist under the carbide-public account. If you are logged in, the gist will be saved under your own personal GitHub account.

Notebooks are represented in a nice human-readable plain text format: each cell corresponds to a different file, with a markdown header file containing all the metadata as a comment.

Example Notebooks

Smiley Face

Draw a smiley face!

Neural Network XOR

Drag a slider to train a tiny two layer neural network to approximate the exclusive or function.

THREE.js Cubes Demo

Many cubes, all floating around your mouse in 3D.

GRAIN Data Cleaning

Import an excel file and clean it up programmatically.

React TodoMVC

An es6 adaptation of the popular TodoMVC example app from todomvc.com.

Creating a Youtube Widget

Add a widget to your notebook that recognizes YouTube video URLs.

Empty Notebook

For people who really like the sensation of writer's block.

Widgets

To try all of these live, visit the Widget Menagerie

Numbers

slider
integer
scrubber

Strings

string
string_diff
color
buffer

Data

chart
surface
ndarray
matrix

Objects

regexp
date
json
json_diff

Basic Widgets

date
html
map
object
react
regex
selector

Miscellaneous

buffer
chart
color
date
html
json
json_diff
map
matrix
object
regex
selector
slider
string
string_diff

Making Your Own Widgets 🎂

Carbide widgets are just react components (what's a react component) with a static class property title and a static member function called match, with the signature match(value, ast). Whenever you open a widget on some expression, carbide runs the expressions value and ast through all of the installed widgets' match functions to figure out which widgets it should show in the tabs. If you have a react component with a static title and match function, you can add it to your notebook like this: 
import React from 'react'
class YOLOWidget extends React.Component {
    static title = '#YOLO'
    static match(value, ast){
        return value === '#YOLO'
    }
    render(){
        return <a href="https://en.wikipedia.org/wiki/YOLO_(motto)">
            https://en.wikipedia.org/wiki/YOLO_(motto)
        </a>
    }
}

doc.kernel.widgets.push(YOLOWidget)

Backpropagation

Numbers

For the mathematically minded, you can interpret the entire program as a function whose arguments are the literals in the program, and whose output is the particular inspected expression. The challenge is then to find an assignment to the arguments, such that the output is as close as possible to the desired value. Once that's done, you can substitute those literals back into the code, and you've assembled a new program whose output at that arbitrary expression is what you told it to do.

In other words, we're trying to solve for the inverse of a function. Another equivalent (more specific) framing of the problem is that we're optimizing a cost function (the distance between the output and the desired target output) over a space of parameters.

Fortunately there's a massive amount of literature available about the problem of unconstrained (or constrained!) minimization. We're currently using a version of NumericJS's uncmin function, which implements a quasi-newton method (similar to L-BFGS), which is a generalization of Newton's root-finding method to multiple parameters.

Quasi-Newton methods need to calculate the gradient of a function at each step to determine which direction to travel toward. Currently this entails evaluating the function several times with slight tweaks to the existing parameters (in the future one could imagine implementing automatic differentiation so one would only need to evaluate once per step).

In case the optimization engine ever gets stuck in a place with zero gradient, the previous working parameters are remembered, so each interaction can start with the closest known reasonable point (We know it affectionately as Guillermo "Memo"'s Memoization).

Strings

While there have been many decades of work done on improving numerical optimization, there isn't nearly as much work done on generalizing the idea to other data structures.

Nested Objects

IM GOING TO WRITE ABOUT THIS REAL SOON NOW

Declared Inverses

IM GOING TO WRITE ABOUT THIS REAL SOON NOW

Cascading Approaches

IM GOING TO WRITE ABOUT THIS REAL SOON NOW

Technology

There's a fair amount of tekmology and stuff that goes into Carbide, and to be totally honest, we totally pulled an all nighter trying to write copy for this website, and there's a lot of detail that we don't really have time to include. But hopefully over the coming weeks, we'll be able to publish some blog posts going into a bit more technical depth.

SystemJS/JSPM

NPM is probably one of the largest and most successful package managers ever created (albeit not without a few kerfuffles). So it was pretty important to be able to easily import modules from NPM.

Originally, we had written our own tiny implementation of NPM which would run in the browser (calling the official NPM API endpoint, locating tarballs, unpacking them in the browser, and loading the files appropriately). Separately, we were using Babel and Webpack to develop the application itself.

But over the past few months we discovered SystemJS, and ended up using JSPM to handle just about everything. In fact, nowadays Carbide's "Current Window" mode uses the same SystemJS loader for both the editor itself and the code which gets executed on it. This means that it'll technically be able to self-host, hot-reload, and instrument various parts of itself. That's kinda neat.

BabelJS

Most browsers don't quite support all the latest features and proposals that people have drafted up, so in order to use the latest and greatest syntax and semantics, people have to use programs called "transpilers" which translate the source code of one language and into another one.

But far more important— as far as Carbide is concerned, is just being able to have access to the abstract syntax tree of a chunk of code, and being able to instrument certain ranges.

Top-Level Async/Await

To make everyone's life easier, Carbide cheats a bit and allows you to use the await keyword in the top level, as if you were writing an async function. The way this works under the hood is basically by detecting top level awaits, and wrapping your entire cell in an async function if it finds them.

We've broken out the Babel transform that does this into its own repo.

Hot Reloading

Hot reloading is really neat, and it's rapidly become a staple of modern Javascript development (to whatever extent you can string those words together and pretend it's a phrase). But it's actually key to creating a fun interactive programming environment— being able to substitute modules without running everything from scratch.

To accomplish this we've built our own Hot Reloading engine on top of JSPM. It's less aggressive in reloading dependents than the SystemJS hot reloader and the Webpack Hot module replacement system. It exploits the fact that in ES6, references are "live" so imports can be updated simply by calling a getter.

However, sometimes you actually do want Webpack's aggressive reloading of dependents— this can be done by exporting a variable __forceReload. 

export const __forceReload = true
To clean up stateful code in preparation for a reload, export a function called __unload that optionally takes the module compiled from the newer code. 
export function __unload(new_module){
    cancelAllEventListeners()
}

CodeMirror

When we started thinking about making Carbide self-hosting, we realized that it had to be made in such a way that it could handle large notebooks. Unfortunately if you use the HTML's natural layout mechanism to lay out your cells, everything still needs to be rendered on every keystroke— there's no way to do optimizations like rendering only the visible lines (because it's then the browser who figures out what a visible line is).

Carbide's current implementation is kind of strange in this regard because we have a single gigantic CodeMirror instance whose document represents the total concatenation of all the cells in the document. We have a special layout algorithm to insert vertical spaces where appropriate so we have enough space to paint on user interface elements on top.

This allows us to take advantage of CodeMirror's neat lazy rendering capabilities and to only render the things which are visible on screen.

Nondeterministic Layout Engine

To Enable Carbidiculous behaviors, like treating text contours as elements that can block floating widgets, Carbide's layout engine uses a form of McCarthy's AMB operator. Essentially, because Carbide need to calculate layout greedily, layouts sometimes have to exist in a superposition of two states, that can only be resolved after further layout calculations. This is sort of like quantum bogo sort for css.

Future

Preview: Adding Custom Kernels

Although Carbide is Javascript-first, parsing, compiling, and running code are all implemented as external modules. This means that it's possible to define a javascript module that replaces the carbide kernel in order to run code in a different language. Here's an example of what a minimal kernel will probably lookl ike: 
export async function parse(code){
    // Return some AST
}

export async function match(ast, code, start, end){
    // Return closest valid expression
}

export async function evaluate(ast, cell){
    // Evaluate the cell
}

Preview: Watlab Kernel

Watlab is compile-to-javascript language inspired by Matlab. Almost all of the syntax is the same as ES2015, with addition of a postfix function application operator, matrix literals, and standard matrix operators (e.g. A\b to solve a system of linear equations).

Preview: Python Kernel

A tiny kernel that communcates with a local python REPL. Pros: filesystem access, cool python libraries. Cons: requires an npm install; npm start command.

Preview: MIT Scheme Kernel

A tiny kernel that communcates with a local MIT Scheme REPL. Pros: filesystem access, arcane wizardry. Cons: requires an npm install; npm start command, arcane wizardry.

Preview: Time Traveling

One of the features that didn't quite make it into the alpha release is a Bret Victor Style program execution scrubber that lets you get a more intuitive sense of stuff like for loops: 
[images]
Stay Tuned.

Halp + Hallu

Send us an email at your favorite address:

Acknowledgements

arbide builds on the work of a lot of people. It incorporates code from lots of open source libraries, but it also owes a lot to ideas from both academia and industry.

Carbide probably couldn't have been possible if it weren't for CodeMirror, by Marijn Haverbeke. Likewise SystemJS/JSPM by Guy Bedford are incredibly useful gems. The uncmin function from Sébastien Loisel's NumericJS works well enough that we consider it one of the blackest of magics.