TerminusDB 4.2 - Encyclopaedia Galactica Release

'This one will take us to Titan'

Today we’re excited to announce the release of TerminusDB 4.2!

Based on the number of new features in this release, it could almost be a 5.0. We focus on the three big developments in the release blow: Optimize, Large Files with Tus, and Console.

Read all about the new release and why we called it Encyclopaedia Galactica.

Get Lit


MLOps

Gavin and Luke appeared on the MLOps Meetup and Podcast to talk about the 'Git for Data' landscape and why revision control for data is so important. Demetrios, the host, knows his MLOps.

Watch or Listen

Watch


Terminators on Tech

Great content on our Twitch channel - Kitty talking about the new console and much more.

Also so really interesting TerminusDB Discussions - Gavin extolling the benefits of central planning for starters. 😀


Terminalia

We celebrated the Ancient Roman festival of Terminalia on 23 February. It was lit. We're going to have to do something much bigger next year. Watch your inbox for the invite.


AI Infrastructure Alliance

TerminusDB is a proud founding member of the AI Infrastructure Alliance! The 25 members of the Alliance represent some of the most cutting-edge companies in the world and together we're building the foundation for data-intensive apps of today and tomorrow.

Press Release


Feature of the Month: Model Building Tool

Data modelling is part of design - and visual tools help you do it well. The model building tool short circuits and democratizes data modelling.


Join Discord


The Star's End Release - TerminusDB 4.0

Hello Terminators!

Happy (and relieved) to announce Version 4.0 of TerminusDB - the Star's End Release*. This is a big step forward in solidifying and extending the functionality of TerminusDB and TerminusHub.

With this release, TerminusDB takes a significant leap towards our vision of a general-purpose tool for data & content management and collaboration.

TerminusDB 4.0 extends the revision control features to allow greater visibility and reproducibility. Provenance, lineage, and compliance are increasingly essential in the management of data and content.

The major additional features are:

Model Building Tool. The database ships with an integrated visual schema building tool. You can visually design knowledge graphs in the same way they are presented to business users. We have tried to make the tool easy to use, while preserving the ability to model the most complex domains.

Document Editor. TerminusDB 4.0 has full surfability, clickability and editability of database documents through the console. This is the beginning of the TerminusDB content management system, which we will be expanding over the coming months.

CSV Manager. With a single click, you can now build a database from a CSV or a group of CSVs. You can then version control the CSVs in Terminus.

Command Line Interface. We are excited to launch the TerminusDB CLI. You can connect to TerminusDB with a Git-like CLI to run queries or use the revision control features.

Read more about the features and what comes next in our release notes.

Download TerminusDB 4.0


TerminusDB Applications

Some great example applications are being developed with TerminusDB.

Dmytri has developed both back-end and front-end ToDo Applications using TerminusDB.

His front-end TodoMVC (feat. Svelte) is more developed and beautifully documented, but we shouldn’t forget the hardworking back-end Todo example. Blog over here on the process.

Here is a video of a recent episode of Terminators on Tech where Dmytri discusses the ToDo work with Cheuk.

Terminator Emmanual Oga built a CMS proof-of-concept using TerminusDB. It is called Neon Tetra, which is now our preferred fish from the characin family.

Leave a comment


Stars on GitHub

We cracked 1k stars on GitHub:

Thanks for all the support! We are hoping to have a stars event at some point in ‘21 - so keep


Virtual Customer Testimonial

Chingying, the VP of Data Engineering at fictional fintech unicorn FetchDough made a short video to talk about how her team gets value from TerminusDB and TerminusHub. The guy drinking the beer at 1.22 has a lot to answer for!

Share


Blogs and Articles

The New Stack wrote about TerminusDB and how we fit into the broader ecosystem

Our friends over at DBpedia featured a TerminusDB blog to announce our entry into the DBpedia association

The good people over at Dolt - our frenimies - even mentioned us in a blog. We are both working to define the Version Controlled Database category.

Product market fit - a blog on something we have been thinking about recently

A beautiful blog about creating a database from CSVs

And finally, a blog on the visual model builder

Join Discord


Terminators on Tech is Back

After a bit of a gap, Terminators on Tech is back. It will be once a month on Thursday at 4pm GMT; 5pm CET; 11am ET; 8am PT.

1st Thursday - Introduction to Knowledge Collaboration

2nd Thursday - World of WOQLCraft

3rd Thursday - Terminators on Tech

4th Thursday - World of WOQLCraft

(in the rare event that there is a 5th Thursday in a month, we’ll have a televised staring contest)

We also have a weekly ‘Issues and Docs Triage’ session in the TerminusDB Discord Server. We’ll gather up any GitHub issues and work out what docs are needed. If you have something you need to say, come along - the squeaky wheel will get the oil.

What: Issues and Docs Triage

Where: General Voice Channel

When: Every Thursday from this week until the end of time @ 2pm GMT; 3pm CET; 9am ET; 6am PT; 7.30pm IST

Visit Discord


See you next time!

TerminusHub - The First Foundation

Hari Seldon was the father of Big Data

Hello Terminators!

It is hard to conceptualize how much happens around here - I am looking back at our last newsletter - which was in August and reads like it came from a different world. I don’t know if any of you have worked at a fast-growing/changing/developing project, but it always feels like the Lenin quote:

"There are decades where nothing happens; and there are weeks where decades happen." ― Vladimir Ilyich Lenin

But all the weeks are the ones where decades happen…


TerminusHub is in the world.

Hub allows you to store databases in the cloud, control who can access them, make sure each person has the latest version, easy sync to update, and can be cloned to any computer complete with complex internal structure and metadata (to our sophisticated in-memory analytical engine).

Lossless, compressed data-structures can be queried "in-place" without decompression shared through the internet. It's a kind of magic ;-)

And as all computing is local (on your machine or in your cloud), the economics are good so we have a generous free-forever tier.

Go to TerminusHub to signup (we have easy Google and GitHub authentication) and download TerminusDB to interact with all of the test datasets that we have provided. You can then start to build your own version control ecosystem.

We have a new electron app for windows users and a soon-to-be-released version for Mac users. We know most people will continue to work with TerminusDB via the Python and JavaScript clients - but it always helps to be able to visit a control console

TerminusHub


The truth

Many ML/AI shops use CSVs + email or slack (or notebooks) for the storage and distribution of both data and features. That creates all sorts of headaches - no version control, just hard to understand suffixes (code in 2001 anybody?); no visibility of what is available or what is shared - just random ‘oh Mary has the output of the GAN on her machine’ information; feature selection is hard to compare and is individual to each experiment, and the CSVs that people share contain no type information so have to be constantly marshaled in and out of formats with repeated cleaning causing all sorts of errors!

Some practitioners try to solve this by using Git itself as a versioning system for data (web scrappers seem to particularly prone to this) - but it is not designed for data, can’t scale with data, can’t hold type information and can’t be queried. We also hear a lot of data-intensive teams say ‘I just work directly on my cloud data warehouse data and it is better to have a single central store’. We think that sounds like code before ‘distributed’ came along. It’s centralized and kills productivity & collaboration.

Here is a hard to read cartoon that explains it from Changying’s perspective (browsers to 200% please):

Share


TerminusHub Video

Here is a video that tries to explain the value prop in less than 90 seconds (made by the excellent Headless Hippies agency - strongly recommend if you need video work):


Joining Things

Since we last spoke, we have joined the DBpedia Association (a project aiming to extract all the structured data from Wikipedia) and the AI Infrastructure Alliance.

TerminusDB took part in the DBpedia Autumn 2020 Hackathon - we tried to enrich the Seshat: Global History Databank data with conflict data taken from DBpedia. It went really well and we produced a step-by-step blog that covers all our work.

You can clone the full DBpedia data set from TerminusHub and run all of the below steps for yourself (for non-technical users - like your correspondent - it is easy to do, here is a video of clone and query in under 3 minutes.

We made a visual at the end of the Hackathon:

You can explore the visualization of conflicts and polities.

*Red - Battle/War/Conflict*

*Purple - polity with a standing army*

*Blue - polity with no standing army*

*Grey - no information about standing army (unknown unknowns)*

*Yellow - known unknown (we know we don’t know)*

The AI Infrastructure Alliance is a group of technologies that is dedicated to bringing together the essential building blocks for the Artificial Intelligence applications of today and tomorrow.  We are currently working to define the canonical stack in machine learning and produced this helpful article to get the ball rolling.

Leave a comment


CSVs

One of the features that we are really excited about is the easy ability to version CSVs using TerminusDB and TerminusHub. We want to be the version control layer for all your data work, so we are making it as easy as possible to marshal CSVs in and out of Terminus. You can automate this process so you just have a backup of versions with commit logs so you can go back to any point in the past with a simple query.

The glorious return of Terminators on Tech was an extensive discussion of CSV versioning with a demonstration included. (Whisper it .xls is coming soon).


Blogs

A selection of the best new blogs:

Size Matters (a look at our approach to compression)

TerminusDB v Neo4j (a side by side comparison)

The TerminusDB Values (η καρδιά μας)

Collaboration for Structured Data

And a video discussion of Esperanto for good measure

Terminus YouTube


Docs

We have a big ongoing project to improve our docs. We had a session in the Discord server to generate titles of all the documentation that would ideally like to see. This is what we came up with:

We are going to have another session in the very near future to actually write the docs - please drop in and give a hand! All contributions will be rewarded with undying loyalty and devotion.

Join Our Discord

Launch: TerminusHub – The Distributed Data Revolution

In the age of data, TerminusHub is arming the rebels to fight back against the internet giants

Dublin, Ireland. TerminusDB today announced the launch of TerminusHub, a distributed data collaboration platform. Powerful versioned data collaboration is a radical departure in the way we manage our data. TerminusHub is dedicated to the democratization of data and empowering people to take back control from the internet giants. It is like GitHub, but for data.

With TerminusHub, you can clone all of Wikipedia to your local machine in less than 2 minutes - and then collaboratively integrate it with another data source to see the links and connections. This functionality is available without cost for all users - TerminusDB is open source now and forever.

The public has been demanding a better solution to collaborative data management for years. With TerminusHub, you can share, build and integrate databases with your team and the wider world. Your project can keep data synchronized and make the latest data accessible no matter where you are. In a suddenly remote world where data is the driver of decision making, TerminusHub is the new paradigm in public benefit data management.

Starting in Trinity College Dublin, TerminusHub is the culmination of 6 years of research and development. TerminusHub has the potential to be a European data management champion to rival the predatory US and Chinese alternatives. TerminusDB was first financed by a €4 million European Commission Horizon 2020 grant.

Kevin Feeney, CEO of TerminusDB, said: “We live in the age of data. Everywhere we look, big data rules our lives. Amazon, Google, Microsoft, Facebook and the other internet giants have too much control, TerminusHub is arming the rebels with advanced data management tools - let's start the fightback”.

In the last 15 years, there has been a revolution in software engineering. For coders, the Git Revision control system changed everything. Distributed teams could collaborate on common code bases. But people who work with data didn’t have an easy and secure way to manage their pipelines and workflows. Until now.

TerminusHub is revolutionizing database collaboration and management.

8 September 2020

END

Loading more posts…