Interesting stuff! Which use cases do you think developers will use you most for?
There could be really interesting abstractions that people might build on top of this. Like automatically creating and animating infographics, making background sounds, or video cutting and recycling. If you spin this 100x further an entire video creation studio might emerge.
Which parts of Video Infrastructure do you want to build first? Which other higher-level parts could be built by you or users? Where could this go?
Thank you! As of now, many people use Revideo to programatically create Youtube videos (primarily entertainment content, think of AI generated memes and Youtube Shorts) and for use cases in video marketing (think automatically generating and A/B testing ads from a product portfolio). As the project becomes more mature and better, we want to enable developers to build more complex, and especially interactive video editors using the framework.
I'm glad you raise the point of reusability and people building abstractions - what I find really exciting about code as an interface for video editing (rather than GUI) is that it makes it much easier to create and share isolated, reusable components and functions. Especially as we are open source, I hope that many developers will build abstractions and share them on Github
Out of curiosity, what do you mean by "Video Creation Studio"? Would you add a visual interface for editing to Revideo?
Maybe not for editing per se. But for providing meta-functionalities to services built on top of revideo. Rendering, for instance, but maybe even reusable embeds, video watch analytics, download links, programatic re-rendering with webhooks and their configuration, etc.
What other functionalities could you offer?
You can use a third party NPM module to introspect the DB and write out types for you. Can’t remember what it’s called off the top of my head, but maybe “pocketbase-type gen” or something like that.
I just did a quick search and struggled to find anything about the performance issues you referred to - can you link something so I can take another look?
Thanks for the suggestions.
From the code, it s easy to see that you create a WRITE transaction which has the side effect of triggering READ transactions. It is also important to understand that if you mix reads and writes you cannot do it well concurrently with SQLite, so every transaction will be sequential and blocking. Looking at the code further I understand that it will not scale well, since a single write can possibly trigger a waterfall of callbacks creating bottlenecks. The problem in your case is that sqlite_update_hook doesn't transport back data and that it is listening for changes in all tables having ROWID optimisation on. So first things first you only need one such callback and a different approach for integrating it in a document abstraction than table name and rowid predicate in a dozen of registered callbacks.
You will unveil more problems with SQLite for this job as you dig deeper. What you really want is fast writes and dumb querying by document id, whereas SQLite gives you an ultra querying suite that you don't utilize but still have to pay with slower writes. This is a classic system-design problem. Just try rewriting your collection and document abstractions using proper document data storage and you will see how less complicated it is.
Hey! Thanks for the feedback. I'd just be worried people might not be using the correct libraries since I think they still provide different functionality.
So exporting one Database that allows people to pick which driver they want to use might be a simple and user-friendly solution (incl. better-sqlite3) which people have asked for.
- https://github.com/haxtra/super-sqlite3 (Fast SQLite library with optional full db encryption, simple query builder, and a host of utility features, all in one neat package.)
I think the super-sqlite3 source might also be an inspiration for the 'driver' topic: "super-sqlite3 is a thin wrapper around better-sqlite3-multiple-ciphers, which extends better-sqlite3 (the fastest SQLite library for node.js) with full database encryption using SQLite3MultipleCiphers. super-sqlite3 then adds its own query builder and other convenience features."
And do check out this user's XRay (JavaScript object browser component) library for your preferred component framework.
In my bookmarks I also found these other related and interesting links:
- https://dgl.cx/2020/06/sqlite-json-support (An article about SQLite as a document database, using the relatively new 'genrated columns' feature of sqlite 3.31.0, which you seem to be using)
Yup! I doubt this project will naturally evolve into whatever you're describing. And that's ok! This project might help people who want to use SQLite like Firebase, with a similar API and experience but without letting network requests increase latency (see: https://news.ycombinator.com/item?id=31318708).
Addressing main points like implementing atomic transactions (read and write operation on a doc) seems warranted since it exists in Firebase as well.
There could be really interesting abstractions that people might build on top of this. Like automatically creating and animating infographics, making background sounds, or video cutting and recycling. If you spin this 100x further an entire video creation studio might emerge.
Which parts of Video Infrastructure do you want to build first? Which other higher-level parts could be built by you or users? Where could this go?