I chat with Ashley McNamara, Johnny Boursiquot, and Carmen Andoh about the process of getting hired, hiring, and job interviews. If people are the most important part of a team, how do we pick who we work with? What’s the process like? How can it better?

Put this in your ears…

Veritone’s aiWARE platform lets customers run AI solutions at a very large scale, and it does it by spinning up as many instances of the cognitive engines required for a given use case to meet the demand and handle the load. When they’re done, they shut down to free resources.

Engine developers don’t need to worry about this process (the platform handles all that for us), they just need to make sure the engine can receive work, process the files, and send the output.

In order to put an engine onto the platform, you must assemble a Docker container and build an integration with the Veritone aiWARE platform and we recommend using the Veritone Engine Toolkit to do so.

The Veritone Engine Toolkit

The Veritone Engine Toolkit does the heavy lifting of integrating with the Veritone aiWARE platform, allowing you to focus on what makes your engine valuable.

The Engine Toolkit works by interfacing on your behalf with the aiWARE platform (connecting to the messaging queue to receive work and send results, pulling data via GraphQL, and more) and makes simple HTTP requests to your engine to process the files.

You just need to listen on an HTTP endpoint (by writing some kind of simple web server), and provide a webhook which the Engine Toolkit will call when the time is right.

The Process webhook is called for each chunk (a chunk might be an image from a frame, an audio clip, or some other kind of file — depending on what your engine can do), and the response your webhook returns will be delivered by the toolkit to the platform.

Using whichever framework or mechanism makes sense for your language of choice, you can extract the chunk file from the HTTP request, along with a series of other data which is outlined in the documentation and previewed on the left.

The webhook will return a blob of data that describes the output for the work that was done during the processing.

For example, if our engine is finding faces, we might output some JSON that looks like this:

{
	"series": [{
		"startTimeMs": 1000,
		"stopTimeMs": 2000,
		"object": {
			"type": "face",
			"confidence": 0.95,
			"boundingPoly": [
				{"x":0.3,"y":0.1}, 
                                {"x":0.5,"y":0.1},
				{"x":0.5,"y":0.9},
                                {"x":0.3,"y":0.9}
			]
		}
	}, {
		"startTimeMs": 5000,
		"stopTimeMs": 6000,
		"object": {
			"type": "face",
			"confidence": 0.95,
			"boundingPoly": [
				{"x":0,"y":0}, {"x":1,"y":0},
				{"x":1,"y":1}, {"x":0,"y":1}
			]
		}
	}]
}

The Veritone Engine Toolkit will post this data via the message queue to the platform on your behalf, and moments later it will be available via the APIs, as well as the applications that run in the platform.

Here is a complete implementation of a sample Process webhook handler written in Go:

• Using just the standard library, notice how the HTTP request provides everything we need (FormValue, FormFile, etc)
• The handler outputs data into the Vendor object, but real engines should return data conforming to the Veritone standard output in the documentation

Testing your engine

Before you deploy your engine to production, you’ll want to test it to ensure that it’s behaving as expected.

It is recommended that you write code that makes HTTP requests to the endpoints you have implemented. This approach allows you to run the code before deploying your engine to production, or even as part of a continuous integration environment.

You don’t need to include every field that the real requests might contain, instead just include the data that you are using in your engine.

Sometimes it’s handy to get your hands on the ins and outs of an API, and you can do this using the Engine Toolkit Test Console which is embedded in the engine binary.

To activate the Engine Toolkit Test Console, we need to run our engine with some special parameters:

docker build -f Dockerfile -t your-engine .
docker run -e "VERITONE_TESTMODE=true" -p 9090:9090 -p 8080:8080 --name your-engine -t your-engine

The VERITONE_TESTMODE environment variable activates the Console, and since it is served on port 9090, we have to expose that with the -p 9090:9090 flag.

Once this is running, open your browser at http://localhost:9090/ to access the Console.

The Engine Toolkit Test Console automatically checks the engine’s webhooks and other required items (like the Manifest file) so you can be sure your engine is going to work when pushed to production.

What about more advanced cases?

If your engine needs to perform some more advanced interactions with the platform, you can still use the GraphQL API from within your Veritone Engine Toolkit powered engine.

The Veritone aiWARE platform documentation often mentions the Payload JSON, which contains details about the task you’re engine is being asked to do. It also contains a token, which is used to authenticate with the platform and it is this token that allows you to access the GraphQL services.

The payload comes into the Process webhook via the payload form variable. It is a JSON string, so you must unmarshal it before you can get at the data inside it.

In Go, the code to do this might look like this:

var payload struct {
    Token string

    // list the other fields you care about
    // from the payload here
}
err := json.Unmarshal([]byte(f.FormValue("payload"), &payload)
if err != nil {
    return errors.Wrap(err, "unmarshal payload")
}
// TODO: use payload.Token

Whichever language or tech stack you’re using, there will certainly be a library or package available that will parse HTTP requests for you, so consuming webhook requests and providing responses shouldn’t be hard.

If you need a little help or inspiration, you can check out the Sample engines from the Veritone Engine Toolkit GitHub repo.

Once you’ve extracted what you need from the payload, you can write your own code to interact with the APIs to fulfil the purpose of the engine.

More information?

If you’d like to learn more about these technologies, please check out the following links:

• Veritone Machine Box Suite — Machine learning capabilities inside Docker containers
• Veritone’s aiWARE platform — Enterprise grade AI at scale
• Veritone Engine Toolkit open source project for integrating cognitive engines with the aiWARE platform

Building an engine with the Veritone Engine Toolkit was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

They repeated the experiment again, but this time they made a deliberate crack in the windscreen of the car, and within a few days, the car was completely burnt out.

The broken windows theory is a criminological theory that visible signs of crime, anti-social behavior, and civil disorder create an urban environment that encourages further crime and disorder, including serious crimes. The theory suggests that policing methods that target minor crimes such as vandalism, public drinking, and fare evasion help to create an atmosphere of order and lawfulness, thereby preventing more serious crimes. — Wikipedia

Each person who damaged the car didn’t really change the state of it very much. The window was already cracked, so somebody just added another crack; no big deal. Compare this to making the first crack in an otherwise pristine car — the difference is significant.

Later, somebody else comes along and does some other small amount of damage to the already-damaged car, and so on until it is a wreck.

If we think about the Broken windows theory when it comes to code and software design, it’s clear that there is great value in the cost of maintaining quality.

If you work on a project that has flaky tests, then you’re more likely to add more flaky tests. If there is a hacky design, you’re more likely to hack more in. If there is a package called utils, you’re more likely to stick more utilities in there. If one of your HTTP handlers uses a global variable for some state, why wouldn’t yours? If anything, we’re taught that our projects should be self-similar and consistent — so maybe we’re even doing the right thing?

Each person who contributes to this mess isn’t really changing the system much, but they are headed towards a fire.

Like in the criminology theory, policing the small stuff and taking the time to keep the quality of your projects high ought to be one the highest priorities of the team. This can include refactoring, developer documentation, good and clear Makefiles, simple package layout, good quality test code, and more. None of these things should be overdone, they should be as simple as they can be (but no simpler, right Einstein?).

Isn’t perfection the enemy of progress?

No project is going to be perfect, and there will always be things we’re not happy with as a team but that “just aren’t worth fixing.” It’s important to be honest (and positive) about this fact. Complaining might be good therapy for developers, but a team deciding to leave something the way it is for the sake of progress is a respectable position to take; and should be a positive experience and an excuse for learning.

Limiting scope is a great way to ensure teams have the time they need to get the important stuff right and this needs to be understood across the entire company (not just the dev team). Maybe our software doesn’t do everything, but what it does do, it does well.

Conclusion

Remember the Broken window theory when you next mark a ticket as ‘done’. Have you just made the first crack in an otherwise pristine window? If so, get some help from the team and take a little more time to refactor it until you’re happy that it meets your standards.

When reviewing code, police the small stuff. Pay attention to the implied decisions and make sure you’re happy with the decisions (overt or otherwise) that are being taken along the way.

References

• Broken windows theory on Wikipedia https://en.wikipedia.org/wiki/Broken_windows_theory
• https://blog.codinghorror.com/the-broken-window-theory/

We started Machine Box because we wanted to open the potential of machine learning to developers of all skill levels, and to teams who don’t have exuberant budgets or armies of people to throw at a problem.

Our customers have built some very impressive and innovate things, and occasionally they have come to us asking, “Ok, we’ve integrated it, now how do we go into production?” — We didn’t have a great answer, until now.

Veritone’s aiWARE operating system is a production ready platform where customers can build workflows around their machine learning problems, and most Machine Box technology is already integrated and ready to use today.

Over the next few months, we’ll be working to more tightly integrate our technologies to offer a seamless experience and provide simple, turnkey solutions to big ML problems at scale.

At Machine Box we’ve been focussed on solving the ML pieces inside Docker containers, but Veritone has a much wider focus and more ambitious goals. Customers can pick and choose the right mix of services, APIs, UIs, and third-party integrations to best suit their needs.

To learn more about Veritone, read about their AI Solutions with Actionable Insights or book a demo to have it all explained in person.

What happens to my subscription?

Don’t worry, it’s business as usual. In fact, we’re working to see if we can take this opportunity to make our Machine Box offerings even better. Stay tuned for announcements on that front over the next few weeks.

Meanwhile, we hope this news gives you the confidence to push ahead with your projects, knowing that Machine Box is now well established and more stable than ever before.

Questions?

If you have any questions or concerns, please don’t hesitate to contact us, or Veritone.

What is Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Machine Box is growing up: Why we joined Veritone was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

Delivering complex software solutions inside a neat container is exactly the promise of Docker, and Machine Box has truly benefited from this.

This article shows you how to spin up Machine Box in Google Cloud Platform, but the technique works for any Docker image you want to deploy.

You will need:

• Google Cloud SDK (the gcloud command)
• Your favourite text editor
• A terminal

1. Create an account and sign in

Head over to the Google Cloud Platform Console and sign into it. You’ll need a Google account to agree to some terms and conditions etc.

2. Create a project

In the project dropdown, select Create new project. For first time users, you will likely be guided to do this with a wizard.

Choose a project name and setup a billing account before clicking Create.

TIP: Keep a note of the project ID which might differ from the project name.

While your project is being setup, you can get the project ready for deployment.

3. Your project files

Create a folder on your development machine, ideally matching the name of the project you just created.

TIP: These files are expected to go into your source code repository, but follow best practices about storing secrets in there.

Create two text files called app.yaml and Dockerfile.

Dockerfile

The Dockerfile describes which image you want to add. In the simplest cast, just specify which image you want to generate:

FROM machinebox/facebox

In the case of Facebox, we also need to include the MB_KEY environment variable, which we can do in this file:

FROM machinebox/facebox

ENV MB_KEY=YOUR_KEY_HERE

TIP: This is just a normal Dockerfile, so you can do whatever you need to here.

app.yaml

The app.yaml file is where we will configure the deployment.

runtime: custom
env: flex
service: default
threadsafe: yes

network:
forwarded_ports:
- 80:8080

automatic_scaling:
  min_num_instances: 1
  max_num_instances: 10
  cool_down_period_sec: 120 # default value
  cpu_utilization:
  target_utilization: 0.5

resources:
  cpu: 1
  memory_gb: 2
  disk_size_gb: 10

# volumes:
# - name: ramdisk1
#   volume_type: tmpfs
#   size_gb: 0.5

This file describes an autoscaling image (between 1 and 10 instances at a time) with a single CPU, 2GB of RAM and a 10GB disk, forwarding the Machine Box (container) port 8080 to 80, so it’s accessible on the web.

TIP: You can dive a bit deeper on what you can do with the app.yaml file by checking out the official documentation.

4. Deploy

Deploy your first version with the following command line in the terminal:

gcloud app deploy app.yaml -v v1

After a while, the image will be deployed and available.

5. Access your container

Head over to https://your-project-id.appspot.com/ to access the image.

In our case, we see that the Facebox console is available:

Introducing Machine Box

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Deploy Docker containers in Google Cloud Platform was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

This guide will walk you through the steps required to deploy Facebox with Digital Ocean.

TIP: Before you start, it’s important to remember that hosting on Digital Ocean comes with a small cost — it’s very competitive, but it isn’t free.

1. Create a digital ocean account

Head over to https://www.digitalocean.com/ and create an account.

2. Generate a personal access token

In order for us to access Digital Ocean’s services via the API, we are going to need a Personal access token.

• Click the API navigation
• In the Personal Access Tokens section, click Generate new token

• Give your token a name, and click the Generate Token button

Once it’s created, copy it and create an environment variable

3. Create a droplet

In Digital Ocean, the machines are called droplets. And we are going to need to create one so that we can run our Machine Box docker images.

We will use the docker-machine command, which should already be installed on your development machine if you have been working with Docker.

TIP: Type docker-machine into a terminal, if you get an error you may need to install it yourself.

Type this single line into a terminal remembering to replace the PERSONAL_ACCESS_TOKEN string with the Personal access token you created in Step 2:

docker-machine create --digitalocean-size "s-2vcpu-4gb" --driver digitalocean --digitalocean-access-token PERSONAL_ACCESS_TOKEN facebox-prod-1

• --digitalocean-size "s-2vcpu-4gb" creates an image with 2 CPUs and 4 GB of RAM — you can use any size you feel is appropriate
• facebox-prod-1 is the name of our droplet

All being well, this will create a droplet. You can verify this by typing:

docker-machine ls

You should see output something similar to:

facebox-prod-1 * digitalocean Running tcp://152.63.254.5:2376 v18.05.0-ce

4. Connect to the new droplet

To connect to the droplet, run the following command:

eval $(docker-machine env facebox-prod-1)

This is actually just setting some environment variables that the docker-machine and docker commands will use.

TIP: You can run just docker-machine env facebox-prod-1 to see what the values are.

5. Start a Machine Box service

Now we have our droplet running and we are connected to it, it is time to run one of the Machine Box boxes.

In this example, I am going to run Facebox — but you can run any of the other boxes.

docker run -d -p 80:8080 -e "MB_WORKERS=2" -e "MB_KEY=$MB_KEY" machinebox/facebox

This is slightly different to how you might run Facebox in development for a number of reasons:

• We are using the -d flag to tell Docker to run this as a daemon (background task) — you can omit this while you’re setting things up and your terminal will stay connected like during development
• We are mapping the box port 8080 (where the services run) to port 80 — this is what makes it accessible on the web without specifying a port number
• We are using the -e flag to set the MB_WORKERS environment variable to 2 — to match the number of CPUs we have

WATCH OUT: If you see an error complaining about the MB_KEY, ensure you have assigned the environment variable with the value from the Machine Box website.

6. Get the machine’s IP address

In the same terminal, run the following command to get the machine’s public IP address:

docker-machine ip facebox-prod-1
152.65.253.2

In a web browser, head over to that IP address to confirm that Facebox is indeed running:

Now you can update your client code to use the new endpoint instead of localhost.

That’s all folks

We have successfully put Facebox into production in Digital Ocean.

Specifically we:

1. Created a Digital Ocean account and obtained a Personal access token
2. Created a droplet to host our box
3. Spun up an instance of Facebox
4. Made the box publicly accessible

Turning the box off

The easiest way to turn a box off is to sign into the Digital Ocean console and access the Droplets section, from there you can Destroy it using the UI.

What next?

Once you are up and running, you might want to think about the following:

• For scale and redundancy, consider using a load balancer to split traffic over many instances of Facebox
• Secure the box using a firewall
• You’ll need to do a little more work to support SSL/HTTPS — probably with an nginx image that proxies the traffic to the appropriate endpoints

More about Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Deploy Docker containers into Digital Ocean was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

An updated version of this post is available on the Pace blog.

There are some great tools out there for A/B testing that let you automatically present a selection of different experiences — whether layout, colors, fonts, or content. They measure which is most effective, and you make a final decision on which one to go with.

But what if we didn’t have to settle for one experience? What if we could use Machine Learning to learn which experience each user is most likely to engage with, and present different experiences to different kinds of people.

Personalised experience

Let’s say we have a list of choices to present to the user. We want to present the most relevant content first so that our user doesn’t get bored and instead engages with our app or website.

The choices could be anything — news articles, songs, pictures, products, tweets, or whatever your particular data is.

The most important thing about applying Machine Learning is to think carefully about your goals — what do you want to achieve? While it can be fun, applying new tech for its own sake is not awesome.

Given these choices, we want to change the order based on the persona of the visitor.

For choices A to F, they would be displayed differently for these three groups of users:

Machine Learning can learn about your users

The first thing we need to do is learn about our users.

We can start by randomly changing the order of the choices and tracking engagement (clicks probably) just like A/B testing does.

The difference is we will also capture some measurable data about the user too, which can start to form a model of their interests. For example, we might know their age, or location, or previous purchasing history. The goal here is to think about what properties might be important when tailoring the experience.

Things don’t have to get creepy — use data that the user knows you have and doesn’t mind you using. And always let them at least opt-out, if not having everyone opt-in.

We might also decide to give the model other inputs, such as the time of day, or even the current weather, if they are likely to influence our users’ decisions.

The learning applies to other users too

The insights we get from our users can be applied to other users, and even to brand new users who have never used our app before.

For example, given the following simple table of data about some users:

Name    City     AgeGroup   Loved
--------------------------------------------------------------------
Mat     London   30-40      The Matrix + 28 Days Later
David   London   30-40      The Matrix + 28 Days Later
Piotr   Warsaw   20-30      The Matrix + Jack Strong
Paweł   Warsaw   20-30      The Matrix + Jack Strong
Bill    London   60-70      Jack Strong + Das Boot

What films would you suggest to Piotr and Paweł?

Given their age group, and the fact that they liked The Matrix, it is probably sensible to recommend 28 Days Later to them.

If a new user comes along with the following properties, which films would you suggest?

Name    City     AgeGroup   Loved
--------------------------------------------------------------------
Peter   London   60-70      ?

Since he lives in London and is in the same age group as Bill, we might decide to recommend Jack Strong and Das Boot.

Of course, in the real world it’s nowhere near this simple and the patterns are likely to be much more nuanced — never mind when you introduce any kind of big data scale.

This is where Machine Learning can do a better job than humans.

Reward the model

Once we can make predictions, we need to track whether they’re successful or not.

Whenever we get something right (like a user clicks a choice, or watches a movie, or buys a product) we will reward the model to reinforce the learning.

Our model will then notice patterns, and make better predictions in the future.

Meet Suggestionbox

Suggestionbox is a tool from Machine Box that provides a Machine Learning model for this very use case. You use a simple JSON API to interact with the box.

Typing this single line into a terminal will download and run Suggestionbox for you (assuming you have installed Docker):

docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" machinebox/suggestionbox

If you don’t have an MB_KEY — which you need to unlock the box — you can grab one from the Machine Box website.

We will use it to build a little demo to show personalization working.

Create a model

While you can create models via the API, it’s much easier to head to http://localhost:8080 and create the model using the Console.

I am going to create a model called Genres, with five choices.

Make predictions

Our model is now ready to start making predictions.

Of course, they’re not going to be very well informed initially, but very quickly Suggestionbox will notice patterns in the rewards, and we’ll start to see the predictions get better and better.

Try the free simulator

Suggestionbox ships with a built-in simulator, so you can actually simulate real user activity to see how the model might take shape. If you want to try this, you can do so from the Console at http://localhost:8080/console.

Wiring up our web page

There are two things our JavaScript needs to do:

1. Ask the model for a prediction, and use that prediction to decide on the order of elements on our page,
2. When the user successfully interacts with an element, reward the model so it can learn.

Assuming we had a user object that contained some relevant properties:

var user = {
    age: 30,
    city: "London",
    interests: ["music", "movies", "politics"]
}

CAUTION: The JavaScript on this page is simple and bearbones — you should use whatever UI technology you’re most familiar with instead.

Make a prediction

To make a prediction, we might do something like this:

This code turns our user object into a prediction request and makes an AJAX request to the /suggestionbox/models/{model_id}/predict endpoint.

The makeRequest helper can be replaced with the $.ajax call in jQuery, or whatever remote data API your UI framework provides.

The results will come back looking something like this:

Each choice is mentioned with an order and a score. The order is what we care about most, because that is the order we need to present the choices to the user.

The score is useful for debugging, but you shouldn’t order by it. Occasionally Suggestionbox will try novel things to see if there is any more learning it can do — in these cases, the first element in the array won’t necessarily have the highest score.

The reward_id values are used to issue rewards if the user engages with any of these options.

Reorder the elements

Assuming we have the elements in a container, we can just append them back to it to control the order.

If elements are being appended to the container they’re currently in, they’ll essentially just move to the end. We can use this to easily set the order just by iterating over the choices:

At the same time, we are going to capture the reward IDs in an object keyed by the choice ID. This will make it easier to lookup reward IDs later.

Reward the model

When the user clicks one of the choices, we’ll call this function which will make an AJAX request to reward the model:

This function just creates a reward request object that contains the appropriate ID (we look it up via our rewardIDs object) and a value which is 1 for most cases.

We make a POST request to /suggestionbox/models/{model_id}/rewards, before going about our business.

The model will learn that it did a good thing when it suggested that choice, and this is how it improves over time.

Full example

See a full example of this by checking out the suggestpage toy on GitHub.

Need help implementing something like this?

Our team hang out all day in the Machine Box Community Slack which you can get an invitation to today.

What is Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Forget A/B testing: Order elements using personalized machine learning recommendations in… was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

It is a sign of the times that in 2018, the UK Government established a new unit to tackle fake news, and every day seems to reveal more about the dirty tricks played by companies like Cambridge Analytica, including deliberately spreading misinformation, to try and influence electorates in favour of whoever happens to be paying them.

This is a problem because it hands more power to those with money — people and groups who already have plenty — and away from ordinary people, who democracy is supposed to serve. Whether our political preferences happen to be served by recent fake news or not, everybody should be concerned about this.

It is also becoming increasingly difficult to doubt that certain foreign states are even interfering in elections and referendums of other countries in a significant way, by spreading fake news to those who are most willing to receive it.

The recent advances in the availability of artificial intelligence presents us with an opportunity to tackle this problem at scale.

Machine learning classifiers

Classifiers are machine learning models that can take a set of examples and learn which class each example falls into. You can then ask it to classify content that it has never seen before, with impressive accuracy.

For example, let’s say we want to teach a model to classify neutral vs biased article titles. We would gather as many examples of each class as we could:

class    example
neutral  China's space lab set for fiery re-entry
biased   10 Reasons Progressive Liberals Should Vote for Trump
neutral  Thousands of violent crime suspects released
biased   Right wing people are stupid
neutral  New £30m fund to help rough sleepers

Then we would use a technology like Classificationbox to train a model based on these inputs.

How the model learns is very complicated, and extremely difficult to understand if you try and poke around at the internal data structures it uses, but we know it works because we can test it by sending in some of our examples without providing the class, and asking it to make a prediction.

The number of these test predictions it gets right is how we measure accuracy.

Who decides what’s neutral or biased?

It is unlikely we will all agree on the classes in the examples above, so how do we decide what’s neutral and what’s biased?

The model is only as good as the training data, and it will indeed be biased based on who gathered that data. It would of course be possible to train a model that thinks heavily biased statements are neutral, and vice versa.

In a real world situation, the answer is probably to have the training data open, so that everybody can contribute to, and maintain it. Otherwise you have to trust the people who trained the models.

Machine Box provides Fakebox, a fake news classifier trained with significant datasets based on common sense classification of news articles. Any articles that could fairly be considered only slightly biased were not included, so the model does a good job in most people’s eyes.

In this article, we will see how we can build our own Fakebox alternative, using our own training data using Classificationbox.

Prepare training data

The simplest way to prepare training data is to create a folder — if you’re doing this as a team, then a shared folder might work best (like DropBox, or Google Drive, or some internal network location) so everybody can contribute.

Create a subfolder for each class, for example:

/training-data
    /biased
    /neutral
    /satire
    /junksci

Then put each example into a text file inside the appropraite folder, like this:

/training-data
    /biased
        biased-example1.txt
        biased-example2.txt
        biased-example3.txt

Balance the classes

One key principle is that each class should have more or less the same number of examples. If you have more examples of junk science than anything else, the model will likely be biased towards that class.

How much data?

The best number of examples differs widely depending on a number of factors, but you should start with at least 100 examples in each class, and go from there.

The teaching approach

The best way to teach the classifier, is to take 80% of your example data and teach it.

In Classificaitonbox, this can be done with a simple HTTP POST request:

POST /classificationbox/models/1/teach
{
  "class": "biased",
  "inputs": [
    {"key": "content", "type": "text", "value": "...content..."}
  ]
}

The slightly verbose inputs array and key/type/value objects exist because you can actually teach classifiers with a range of different data types, including numbers and even images.

Then take the remaining 20% of the data, and ask Classificationbox to try and guess which class it belongs in. You do this using a predict request:

POST /classificationbox/models/1/predict
{
  "inputs": [
    {"key": "content", "type": "text", "value": "...content..."}
  ]
}

Notice that we do not provide the class in this request.

Classificationbox also supports multiple languages, which you can specify by using the language code in the type, for example test_sp for Spanish language.

Meet the textclass tool

The textclass tool is a simple Go program that walks a folder structure like the one described above, and performs the teaching of Classificationbox for you.

You can install it (assuming you have Go installed) by popping this into a terminal:

go get github.com/machinebox/toys/textclass

Run Classificaitonbox

You can run Classificationbox for free by going to your terminal, and doing:

$ docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" 
         machinebox/classificationbox

• You’ll need to have Docker installed
• If you don’t have an MB_KEY, you can get one for free from the Machine Box website

Train and validate

In a terminal, run the textclass tool:

textclass -src /path/to/training-data

You will be prompted a few times to hit Y to confirm what the tool will do:

Classes
 — — — -
fake: 300 item(s)
real: 300 item(s)
satire: 300 item(s)

Create new model with 3 classes? (y/n): y
new model created: 5abe0d3302484439
Teach and validate Classificationbox with 720 (80%) random items? (y/n): y

After some time, you will be presented with the results:

Validation complete

Correct:    173
Incorrect:  7
Errors:     0
Accuracy:   96%

So we now have a classifier that can make predictions, with 96% accuracy.

Put into production?

If you want to put your model into production, you can — since Classificationbox is just a Docker container, you can save the state file, and use it when spinning up new instances in your own environment, or in the cloud.

If you share the same state file with multiple instances of Classificationbox, you can load balance the traffic to achieve planet scale.

Need help?

We hang out all day in the Machine Box Community Slack, and you’re invited to join us to ask questions, or tell us about what you’ve built.

What is Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.

Detect fake news by building your own classifier was originally published in Machine Box on Medium, where people are continuing the conversation by highlighting and responding to this story.

Someone: How would you sort this list?
   Them: I don’t know
     Me: How would you find out?
   Them: I’d probably google it, or look on Stack Overflow

Correct answer; we offered them the job.

Online communities need to be welcoming

Using online resources like Stack Overflow is a great way to solve problems and learn things that you need to know for your particular task in hand, not just for beginners, but especially for them.

But sometimes, they can be harsh places to be.

Yesterday April Wensel tweeted some helpful alternatives to some pr*ckish comments made on Stack Overflow:

April Wensel on Twitter

This was just posted by a guy with 100K+ rep on a beginner StackOverflow question. EQ note: Even this kind of subtly condescending comment can be very discouraging. The words "exactly" and especially "clearly" are warning signs in this context.

April has a rare superpower of somehow being able to communicate the same information without sounding like an absolute _________ (insert your own word there).

April Wensel on Twitter

Consider this easy alternative: "In order to help you find the error in line 49, we'll need to see the rest of the code. Could you add it to your question?

I realise the irony in childishly calling this person names, but there is a serious point underneath. People new to tech are already overwhelmed by the sheer size of the knowledgebase. They tend to assume that senior tech people know everything, and that makes them feel like they’re ever further away from where they need to be.

The truth is that senior people in tech only know a tiny fraction — but they probably know how to figure things out. One of the best ways is still through communities like Stack Overflow, and if they’re to have any value at all, it’s vital that these places remain open and welcoming to everybody.

If we give the anonymous commenter above the benefit of the doubt, we might conclude that they don’t realise how their comments come across, or maybe they’re one of those people who prides themselves on being blunt and forthright at all costs. Either way, shouldn’t we use technology to help us be better humans?

I replied with this joke:

Mat Ryer on Twitter

@aprilwensel Let's collect examples of courteous and helpful comments and examples of 'the other kind' of comments and train a classifier. People can use it to test if their reply is likely to make them look like a prick or not. #JokingNotJoking

But then I realised that you could actually do this pretty easily with Classificationbox.

Classificationbox is a build-your-own machine learning classifier in a Docker container.

Your models are only as bad as your training data

The tricky bit is finding the training data. We’d need to figure out a way to collect enough examples of the two classes of text in order to teach a model to be able to tell the difference.

I’m sure there are some smart ways of scraping this data from the online communities, or in the worst case we could manually collect it — perhaps as a community. Either way, the goal would be to get about 500 examples of each.

Classifiers like Classificationbox work better when the classes are balanced (i.e. they have the same number of examples) so it doesn’t get biased.

You should also train the classifier in a balanced way by distributing the order in which examples are taught, rather than teaching all the examples from one class, followed by all the examples from the other.

Teaching

Machine Box is a little different to traditional machine learning technology in that it allows you to teach in a very light an inexpensive way, on CPUs rather than GPUs, and continuously — so you can improve the models over time without any retraining steps.

First we’d create a model with a request like this:

POST http://localhost:8080/classificationbox/models
{
  "id": "s1",
  "name": "Sentiment",
  "options": {
    "ngrams": 1,
    "skipgrams": 1
  },
  "classes": [
    "pleasant",
    "unpleasant",
  ]
}

The classes array lists the two classes of comments we are going to teach, while ngrams and skipgrams can be used to fine tune the language processors.

Then, to teach a pleasant example, we’d make a request like this:

POST http://localhost:8080/classificationbox/models/s1/teach
{
  "class": "pleasant",
  "inputs": [
    {"key": "comment", "type": "text", "value": "comment here"}
  ]
}

And an unpleasant one:

POST http://localhost:8080/classificationbox/models/s1/teach
{
  "class": "unpleasant",
  "inputs": [
    {"key": "comment", "type": "text", "value": "comment here"}
  ]
}

The inputs are just a slightly complicated way of saying that the comment text field value is comment here.

The additional complication comes from the fact that Classificationbox supports a myriad of data types:

Making predictions

Once we’ve trained our model, we could then use it to make a prediction by providing a new set of inputs:

POST http://localhost:8080/classificationbox/models/s1/predict 
{
  "limit": 1,
  "inputs": [
    {"key": "comment", "type": "text", "value": "new comment here"}
  ]
}

The model will (surprisingly quickly) do its best to predict which of the two classes this new comment belongs in, based on what it learned from our examples.

The response gives us its answer:

{
  "success": true,
  "classes": [
    {
      "id": "pleasant",
      "score": 0.94
    }
  ]
}

In this case, we can see that the model considers the comment to be pleasant.

Tech people could use this model to help them figure out if what they’re about to post is pleasant or not.

Thought police?

Like with any technology this can be used or abused. As a firm believer in freedoms of speech, I would hate to see this technique employed as a moderator, blocking input that didn’t adhere to its standards, like some kind of snowflake terminator. In the case of children I could see a good argument for doing this, or at least providing it as an option for parents.

But people who are able to communicate without upsetting people are often better communicators, and who doesn’t want to be a better communicator?

What is Machine Box?

Machine Box puts state of the art machine learning capabilities into Docker containers so developers like you can easily incorporate natural language processing, facial detection, object recognition, etc. into your own apps very quickly.

The boxes are built for scale, so when your app really takes off just add more boxes horizontally, to infinity and beyond. Oh, and it’s way cheaper than any of the cloud services (and they might be better)… and your data doesn’t leave your infrastructure.