The concepts of forking

Lately there has been a lot of discussion about “hard” or “soft” forks related to MySQL. As someone who has done a successful fork of MySQL, I think this is both confusing and trivialising the concept of forking.

In my previous blog,  I did touch a bit on this topic, but it looks like some more clarifications are needed.

When we did the initial fork of MariaDB from MySQL, we tried our best to keep things 100% user compatible while still adding new features and fixing issues in MySQL. For MariaDB 5.1 -> MariaDB 5.5, we merged all relevant changes from MySQL into MariaDB.

This did not mean that MariaDB was 100% compatible with MySQL, as any change in a fork makes things incompatible in some manner. For example, the enhanced optimiser in MariaDB 5.5 did work slightly differently (better) than MySQL, and if one used any of the new features in MariaDB, one could not trivially go back to MySQL anymore. However, for most users these changes were not notable and allowed most Linux distributions to automatically move MySQL users to MariaDB without any disturbance.

Over time, the merging of MySQL code became harder and gave us less benefit compared to the effort of doing the merges. The new MySQL developers had started to move source code around (which made merges harder), and we, the MariaDB developers, were not happy with the quality of the code related to bug fixes or some of the new features. It was easier to write the new feature from scratch than to use the MySQL code. However, for each feature we did our best to ensure that the syntax and behaviour were identical to MySQL.

Another big problem was that MySQL started to copy features (not code) from MariaDB, but used a different SQL syntax than what MariaDB was using. One example is the usage of CHANNEL in multi-source replication. It did not make any sense for MariaDB to copy the multi-source code from MySQL, as we already had a working, stable implementation we were happy with.

With MariaDB 10.0, we decided to stop merges from MySQL and instead monitor new features and implement those that we thought made sense for MariaDB.

Moving to MariaDB 10.0 allowed us more flexibility in adding more features to MariaDB without being constrained by the MySQL code, like Galera, Oracle compatibility, and a lot of other things listed here.

Nowadays, most of the MariaDB development work is adding features customers and MariaDB users are missing (link to MariaDB 13.0 roadmap will shortly be added here). A lot of this work is related to new Oracle compatibility required by new customers, like FULL OUTER JOIN. There are still a few notable features in MySQL that we have not had time to re-implement, like multi-value indexing (for indexing JSON), JSON operators, and LATERAL tables. All of the mentioned ones are on the MariaDB 13.0 roadmap.

We, the MariaDB developers, are still working on keeping MariaDB compatible with MySQL (and Percona Server). In MariaDB 10.11, we added support for the popular extensions from Percona Server. In the latest MariaDB versions we have ensured that one can replicate from MySQL to MariaDB and back.   We have also added support for the caching_sha2_password plugin, to allow MySQL users to switch to MariaDB without changing their passwords, support of the default MySQL character collation set, utf8mb4_0900_* and multiple JSON functions.

We also listen to MySQL users moving to MariaDB and do our best to implement the features they need to be able to move to MariaDB. The MariaDB Foundation is there for those who want to be part of this effort!

The above hopefully gives the needed background to discuss different kinds of forks (just kidding) in more detail.

Internal fork

• Fork where the company/original development team forks the product for political, redesign, or development reasons. The fork may be more or less, or not at all, compatible with the predecessor.

Examples:

• MySQL 8.0 (someone could call this a “hard” fork as it was hard to move to it and very hard to go backwards )
• OpenOffice → Apache OpenOffice (after Oracle acquisition; internal governance shift)
• Sun Solaris → Oracle Solaris (post-acquisition direction change)
• KDE 3 → KDE 4 (often cited as an internal “hard” break due to massive architectural changes)
• Python 2 → Python 3 (not a fork in licence terms, but functionally an internal compatibility break)
• Drizzle (https://en.wikipedia.org/wiki/Drizzle_(database_server)

External fork

• When an external group or company forks a project for various reasons. The most common reasons are creational differences in how to take the project forward or distrust in the original project owners.

The external fork has a lot of subcategories:

Downstream "no-changes" fork

• The fork is based on the original project with a small, limited subset of changes to get the project to work within an ecosystem or with an external/internal project that requires some minor changes.
• The code is basically a rebase plus patches on top of the original code.
• No user-visible changes from the original project.

Examples:

• Packages in Linux and other OS distributions
• Ubuntu kernel (downstream of Linux with minimal, policy-driven patches)
• Homebrew / MacPorts packages
• Debian-patched GNU tools
• Android Linux kernel (arguably borderline, but many devices are close to upstream + patches)

Downstream fork

• The fork is based on a rebase of the original code, but with user-visible changes that bring a different user experience while keeping the base 100% compatible with the original project. It is reasonably easy to move to the fork, but harder for users of this fork to move back to the original.
• The forks usually have the problem that newer major versions have to drop options or features when the original project adds them, which makes upgrades to the next version a bit harder.

Examples:

• Red Hat Enterprise Linux (downstream of Fedora)
• Ubuntu (downstream of Debian)
• Amazon Linux (downstream of RHEL/CentOS lineage)
• PostgreSQL distributions (EDB Postgres, Amazon Aurora PostgreSQL-compatible)
• Percona Server
• MariaDB 5.1 -> 5.4 (these MariaDB versions never had to drop a feature)

Compatibility fork

• The fork was originally a 'Downstream fork' but moved to, instead of using rebases, only merging selected patches from the original project and rewriting things the developers disliked. The goal is still to have high compatibility with the original project.

• Examples:
• LibreOffice (from OpenOffice.org)
• Jenkins (from Hudson, especially post-Oracle divergence)
• Percona XtraDB Cluster
• MariaDB 5.5

Independent fork (or "branch")

• The fork is no longer dependent on the original project. It may still take selected patches or ideas from the original project.
• It usually tries to keep things compatible to make it easy for original project users to move to the new project, but the main focus is solving new problems for its growing user base.

Examples:

• GhostBSD
• OpenBSD (from NetBSD)
• Illumos (from OpenSolaris)
• systemd (initially replacing sysvinit, now fully independent ecosystem)
• Neo4j Community vs Enterprise split (conceptual fit)
• Firefox (historically from Mozilla Suite)
• MariaDB 10+

Some people have recently expressed that they are afraid that MySQL development is stopping or slowing down, and others have started to talk about the need to do a “soft” fork of MySQL.

The point I am trying to make is that if these worries are real, then any fork will sooner or later have to become an independent fork/branch or die together with MySQL (as there will be no new features in the fork).

One of the mantras in open source is that it is better to join an existing project than to create a new one! Instead of talking about creating yet another fork of MySQL, it would be better if everyone gathered around MariaDB! MariaDB development is not dependent on Oracle for its future. This is assured by the MariaDB Foundation, which was created to make it easy for anyone to participate in the development of the MariaDB server. MariaDB plc is working together with the MariaDB Foundation to make this possible.

MariaDB is, after all, created by the same people who created MySQL and is developed in the way it would have been if Oracle had not bought MySQL. The rapid adoption of MariaDB (350+ million database installations and rapidly increasing) shows that MariaDB is truly the future of MySQL.

PS:

Please leave a comment if you have a better name for any of the fork categories, another fork category that should be added, or more examples for the categories.

Press release concerning Oracle/Sun

FOR IMMEDIATE RELEASE

MYSQL FOUNDER OUTLINES SOLUTION:
INSTEAD OF LETTING SUN SUFFER,
ORACLE SHOULD SELL MYSQL
(ORCL, JAVA)

Michael 'Monty' Widenius says European Commission is "absolutely right to be concerned" about proposed merger between Oracle Corporation [ORCL] and Sun Microsystems [JAVA], nominates award-winning EU strategist to support the proceeding

Tuusula, Finland, 19 October 2009 -- Michael 'Monty' Widenius, the creator of open source database MySQL and founder of the namesake company later acquired by Sun, today suggested Oracle should resolve antitrust concerns over its US$7.4 billion acquisition of Sun by committing to sell MySQL to a suitable third party. The proposed takeover has not yet been consummated because it is being investigated in depth by the European Commission as well as competition authorities in several other jurisdictions.

Widenius, who posted this press release to his blog, believes the EU's antitrust regulator is "absolutely right to be concerned" and called on Oracle "to be constructive and commit to sell MySQL to a suitable third party, enabling an instant solution instead of letting Sun suffer much longer."

The Finnish software developer and entrepreneur wishes Sun "all the best, but MySQL needs a different home than Oracle, a home where there will be no conflicts of interest concerning how, or if, MySQL should be developed further."

MySQL was the only Sun business unit to be mentioned in the EC's early September announcement of its in-depth investigation into the proposed takeover.

Acquirers commonly resolve regulatory concerns (before, during or after an investigation) by committing to divest problematic assets to a third party. By contrast, Oracle and Sun officials have thus far insisted they continue to seek approval of the entire transaction, irrespectively of Sun currently losing, according to Oracle, $100 million a month.

In order to support the regulators' work on the case, Widenius' new company, Monty Program Ab, works closely with Florian Mueller, a MySQL and EU affairs expert. Widenius said: "Florian gave MySQL strategic advice from 2001 on and was a shareholder until the sale to Sun in 2008, and with our support led an award-winning campaign against a proposed EU law on software patents. In August he helped us to demonstrate to the EC the need to investigate this merger and he is now on board again to meet the information needs of regulators, journalists and analysts."

According to Mueller, "every day that passes without Oracle excluding MySQL from the deal is further evidence that Oracle just wants to get rid of its open source challenger and that the EU's investigation is needed to safeguard innovation and customer choice. This is highly critical because the entire knowledge-based economy is built on databases."

Mueller demands more respect for the EC: "It's inappropriately arrogant for some interested parties to suggest that the EC has yet to understand the case. The EC is really doing a great job under huge time pressure."

In what he calls "a solution-oriented information effort that is now necessary after other parties made public statements on the case in recent weeks", Mueller announced that he will be available to journalists and analysts in Brussels (Wednesday, 21 October), London (Thursday, 22 October) and Silicon Valley (Monday, 26 October) to discuss the case.

In August, Mueller authored a position paper that Monty Program provided to the EC along with several other submissions. The latest version of the document was published today on the Internet.

About Michael 'Monty' Widenius and Monty Program Ab

Michael 'Monty' Widenius is the creator of MySQL, the world's most popular open source database. In 2001, he founded the namesake company that was acquired by Sun Microsystems in 2008 for a total consideration of approximately US$1 billion. The European Private Equity and Venture Capital Association (EVCA) named this transaction the "European Venture Capital Deal of the Year 2008". On a previous occasion, Widenius had been named the Finnish Software Entrepreneur of the Year 2003.

A visionary leader of the open source community, Widenius created MySQL's dual-licensing business model together with co-founder David Axmark. MySQL became the first piece of software to be available alternatively under a commercial license or the Free Software Foundation's GPL.

In 2009, Widenius left Sun and created a new company, Monty Program Ab, based in Tuusula (Helsinki area), Finland. Monty Program Ab develops MariaDB and the Maria database storage engine and other MySQL-related technologies. The company is a founding member of the Open Database Alliance.

Monty Program Ab corporate website

Michael Widenius' blog

About Florian Mueller

Florian Mueller is a software industry veteran with 24 years of experience (starting as an author at age 15) as well as an award-winning EU policy strategist. Previously founder and CEO of a startup he sold to the Telefónica group, Mueller became in 2001 an adviser to MySQL's then-CEO on corporate strategy and held shares in the company until its sale.

In 2004, Mueller created a campaign in 17 languages against a proposal for European patent legislation, finally rejected by the European Parliament in a historic decision at the end of a bitterly contested process. The Economist Group's European Voice named Mueller the EU Campaigner of the Year 2005 (a prestigious award that went to Pope John Paul II in 2002 and Governor Arnold Schwarzenegger in 2007). Managing Intellectual Property named Mueller to its list of the 50 most influential people in intellectual property (2005 and 2006). In 2005 he also received a CNET UK award (Outstanding Contribution to Software Development) and made it to the list of Silicon.com's 50 "Silicon Agenda Setters".

In 2007, Mueller successfully defended the EU-related interests of Real Madrid CF, the world's most famous soccer club with approximately 200 million fans worldwide.

Contact data

For further information concerning this news release, please contact Florian Mueller (telephone: +49-171-2632226, email: florian.mueller@live.com).