v0.1
psviderski/uncloudv0.1Jan 24, 2025by psviderski
AI Summary
This is the initial v0.1 release of uncloud, a distributed container orchestration system that provides a cloud-like experience with self-hosted infrastructure. The release includes a complete cluster management system with Corrosion-based distributed storage, Docker container management supporting both replicated and global service modes, gRPC communication, and WireGuard peer-to-peer networking.
Key Highlights
- Initial release of uncloud - a distributed container orchestration system with cloud-like experience
- Corrosion-based distributed store for cluster state management with subscription support
- Docker container management with support for replicated (1 replica) and global service modes
- gRPC-based communication with transparent proxy routing requests to local machine APIs
- WireGuard networking for peer-to-peer cluster communication with automatic peer endpoint rotation
New Features
- Service management: run, rm, inspect, ls commands with --mode flag for replicated/global modes
- Port publishing with PortSpec type supporting host and load balancer formats
- Machine management: init, list commands for cluster participation
- Docker integration: PullImage, CreateContainer, StartContainer, RemoveContainer, ListContainers methods
- Corrosion distributed store with containers and machines tables
- gRPC proxy system for routing requests to remote machine backends
- Landing page with documentation, demo, and email subscription
- Installation scripts (install_cli.sh) for easy deployment
- e2e cluster lifecycle testing framework
- ucind (Uncloud in Docker) for running machines in containers
- Caddy reverse proxy integration for HTTP(s) service routing
- Exponential backoff for network errors and automatic subscription resubscription
Full Release Notes
## Changelog
* df25b0d0989f23f8f86213a173fb6aa2e508c0c9 add --mode flag to run command
* 2a898b3eee397d57efd2a27bcad245173258b51e add -v|--volume to bind mount host path into a service container
* 5cb765d28665940fb3052ee9224f62989a71db8b add Docker gRPC service to create and start a Docker container on machine
* b23699470b22429c29a079ad56c9e82018561642 add EmptyResponse common message
* 708e14a4e0c237898776907967aa1ece9402b111 add Exec and ExecMulti methods to corrosion API client
* 6d39174b519dbf0e747bdbdce54d70a286c5f553 add Healthy method to Container that derives it from status string
* 16120b80ba1c86ee6982ec58272f9725fe85a0db add ListContainers and RemoveContainer methods to Docker service
* 1fb8a3d7a0adc15470f0197cac8a868068f00aa5 add ListServices method and ls CLI command
* 5621a84e9917eb3d41349b78789fd2931cfc528c add Mode to Service entity
* 43ddb5c363662c1f1fbc967511f2ec7c017685f5 add PullImage method to gRPC Docker service
* 4de7ecd74708bf0431c3c241f082f4a0037dad7d add String() method for PortSpec to convert to publish format
* 2134f3c1d98d2f49840b9847a12427e1fa087e1f add a stub for Caddyfile controller
* 6d1c3aa5b234be169a4d8f18c648e8a1ea7e44df add basic client TCPConnector to directly connect to a gRPC endpoint
* 349dd0d2efbc26fd68a6d76c261f95fe0ca9fb15 add basic e2e cluster lifecycle test
* bfbcec34e8067843004e7f09c8ad9a0cfb6245ba add caddy/v2 dependency to generate a config
* c6358474acf441d1da6111be6522b358c23f59bc add comments
* c2b466b046248fe33996f82df46d55efdce7a31c add common Metadata message to be injected in reply messages by gRPC proxy
* 73d41c75b3865a0a7e72ce3b71fdb399081c678b add compose-like progress for running service
* f7c4490c103ef21364c87e00c700c05a5fd8d449 add diagram for landing page with drawio source
* 5504951a8aaa7971db70c31a08538eca0bf9204a add e2e test for global service creation and removal
* e123f7214e71982539c0bb2608c54f8c1cf21aaf add exponential backoff for network errors in corrosion client
* c29fa571e1d6cec231a73fa776504cf2fe95b70f add favicons
* 34316c77cf9327fae02c17452ca3f79b8996088e add helper Empty message
* 8e53fd46fa3e59eeee7d70bcd69ba503b3e6f81f add how it works section with diagram
* 6e0df53f13c9d60cbe77ef5f848e46a5deca037e add in docker service runner for corrosion
* e73e38957a1f95eaf5ed9daca8040c592b8ef232 add index on containers.machine_id
* 4ec87db8e8616476e17a0326a13048e2c4beae79 add init machine to store when initialising a cluster
* bda1123ae69a70817b61a6dd98c27d28787d1a46 add install_cli.sh script to be donwloaded at get.uncloud.run
* 5061097dfff5b798a28a7d4d63fa5edba5572cdd add landing section to follow the journey and subscribe via email
* bb967461d080fd9d437b66c04ea65ece72976687 add logging for starting/restarting corrosion service
* fa54584128f7b92b9cf49bdbb4979d2d731dd023 add logo and project name
* 085ee285e8c2c5ce8db2a815f317d3f23199a756 add make targets to build and push corrosion image
* 24a824b8900384e3cdec287f32e019e4633f0c61 add make targets to build and push ucind image
* 3c1d2201ce684bb8c89d4eaf17b9fc86368a2090 add methods to create, update, list, delete containers in store
* 336bdf9f4bed6d9e5bd8522a297099cab7fef521 add minor log
* e5bda92ec6e215e53551ab9f799f43c65ecf47e1 add opptimistic sleep on corrosion service start to wait for schema initialisation
* daada3bbe9c7df1971c836190305ea607b30222f add service inspect command, introduce squirrel for sql query building
* f85b5dd01d60d30cb153c6af2b9677e60668f613 add service rm CLI command
* 8db7ac84a08a9f42e2927971065771356631ce8c add subprocess corrosion service runner
* 3d5fe0a82482d74311fa1df3d902e3b5a4f072ec add support for tcp cluster connections in uncloud config
* 2819ba63aa3a8d56ace46351903851c19c584762 add sync_status field to containers table
* b0162528217c8d48b83ccfa1f498edcd9eb75f90 add terminal demo to hero section
* d7bad5ed46676d3d6b6ca782ce7aeded97025366 add todo
* 40f929847cb5eeb5fd11c8eafc10d7c3682f9514 add updated_at field to containers table
* 848abbd2f893ffb5d2d26af849b058c447a6bf5b add validation for PortSpec
* cbe7c563cd1e4490c43baddb3be8a72a8dfc1d17 add value proposition section: Cloud-like experience you control
* 9755a6973d8b68589acb5ab4f1d6dd23a4fee87c allow one2many for ListContainers
* 5c74b09f455139fa6fbcbb71bccc80aeaf4f7fc2 allow running a service on a specified machine
* 94186b2ae70b862ac8f79fdfdb344524fb0a31ce allow to subscribe to machine changes in the cluster store
* 1bcb8836efff767b5f42d25a84b1e4e7deec06c2 auto resubscribe a corrosion subscription if an error occurs
* ee580df6afc59a02188b77e4789880ab32c8ba8f automatically rotate WG peer endpoints if can't establish a connection
* 8bfb3b80ed6a394d93f7d044ed290d697492e4aa basic validation for hostname in port publishing
* e9d07687f22c1989bd4c5b83a0f621dd2e4dea31 build corrosion image based on wolfi]
* 7b98ed34df42d00874c970df9c9eba701fe53507 calculate WG peer connection status
* b25938acc11ac875284421310070f5c28bc3e89b clean up corrosion systemd service
* e82b78df49952ca56e48c5b16eb8e3a93a028884 comment DB schema
* 911522c38805676b6c7dcebb62782fee4b55234d comment out alpinejs
* 69c91673552410070af4794cb926013f70cb6eab configure corrosion bootstrap peers when initialising a machine
* 741378d0b7418d71c7f705e8341c7fb794a31962 create containers table in Corrosion store
* 9fcc319c3a6ad26691ef8a8506d262e023ba85fe create corrosion config
* 0effba0adb3b0d33e3b2743485b89db38012931b create corrosion store when initialising machine
* 4479490db246c91b9e415c638df142006d907114 create custom NewSlogTextHandler log handler to override the json one patched by caddy pkg
* 26df6740e32433808d67c62faa6b03b4f729cd0d define PortSpec type for publishing ports
* c3ba43a8fa19c9f45e93da8060fc7ce8b251a834 delete old non-distributed cluster state, fix storing endpoints in store
* e851f25b123a8415bb23cf5f44eb782ff7be6b0b delete removed Docker containers from store, sync only uncloud labeled
* 3ee7cb6e8e7d4edd2030faa8ed9ecb13b9a2ada8 describe features on landing
* 9dbcf1e902e9f3277f399f349fb5f11f07ada77a do not build and deploy uncloud binary to dev machines
* aa236aac10fc9a8454876d02e9d32251cbfda1c1 do not init corrosion data dir in install.sh
* 2f6c988125b26500c36dc476fd2139a12d6d368b do not rotate endpoints on daemon restart, handle endpoint changes on established reverse peer connections
* f2e730796f908f3156e10d948688dcd53d4c8e45 enable Kit email subscription form
* 0652123bc7a28dd6a895aa56987328624a5a7625 encode and decode published ports using container labels
* 5fe77c688b9b6b57661621e5d342df17a9d46320 exclude Docker bridge br-[0-9a-f]{12} interfaces from machine routable IPs
* a4be45186d49e20989b2415cf5572fc2379c31c3 find first available machine for running a service
* 3f7726e5fb6b21f061a9694359d83a929e594e17 fix SlogTextHandler to respect minimum log level
* bee101aa3eaa1823dabca99e3589d959e80df4e4 fix asset template in .goreleaser config
* 386e4e25f3243f9ba02ea1e23b3aad235f9d0f8b fix context in PullImage
* 6f9fc69c2ed80aa1e0f621352cb097bf02d15d2a fix corrosion data dir, add reload to systemd unit
* c0dff563784f78a9be3b9d463adc3162222f772a fix filter unmarshalling for list containers
* c27527361717ec65d3b5a8d1aad9e9de53f5921c fix install.sh to start uncloud.service after installing all systemd units
* 092fe254aa561080cd33ebab10aeeba45af46650 fix navbar on mobile
* 0b5dd2f7ba71a7a6e6ab45936c7bdbdc69ef61d7 fix network peer configuration when just joining the cluster before store is synced
* e6df331fb8bc727b0ca76f351661c1b936be19f0 fix options for machine list command
* 2c0cf2850931cf221998ad46896f5bac107598ba fix peer status persistence, update status every second
* 042b44cbb6dad0bfac84881ad9903456fa337bac fix port binding for host ports
* 1316dd3ffc4824147b8af90cc81fd6e711cc7742 fix proxy Director usage and close on shutdown
* 3ce9c8be042c4af9200e01d087b7b9cee30b6b0a fix run command, support 1 replica and global mode
* 190d0de61fd7bed79e1c342f348b177f4a383fbd fix warnings on landing
* 2fd1ee203ab96abb48c913391a2ade72c875a573 fix: access to /run/uncloud.sock for a non-root user when initialising a fresh machine
* 2eff7d1e22d27a3d630989f97c252b4c98747821 format
* bb77ff9246b0c723b57331dd45c8ba846d4a7cbd format
* 8a3f0032aec648b9c8261e3df0401891fa5f6692 format machine command flags
* 11612a802ccda6f3dd7e34269535f78653422173 forward NotFound errors from StartContainer and RemoveContainer
* 5c96f75958ea948ded77d30b84f59e9e5d8c627e generate a name for the service
* 7556f8cb124dd66fc196945e70e04114a7eed224 generate basic caddy.json with empty http and https servers
* 2b3f325075118edeabfec897c14d08ccb96434c7 generate caddy config with routes from published http(s) ports
* 035601d3662b026f27e63e3ec9b84caf009b6a4c generate container name and assign service labels to container
* 6951221b883cad74598136ffd14332cbc9b7702f github and get started links in navbar
* cedafd069df53e986de49255f983bf34f2f1cb70 implement ClusterMembershipStates for corrosion admin client
* 9cb4b35546c5b06450b4db3c6cc7665527f99d65 implement PortSpec parsing
* 6d4698fc1de8a758ba4ac941b65e1c053e2656ab implement Query in corrosion API client
* 88ce2d51810f8b375cc8b43e1e4aa6f558f7e4cf implement RunService for replicated mode with only 1 replica
* 2727c3c97ec211d5767103eb71b42f4e34276b7a implement RunService in global mode and RemoveService
* 93dfa5ca9dc1125eda45729f52a283bd618b2e07 implement admin client for Corrosion admin API
* 99c9aabdfdfd58ab282b32b0842faf8aa2b5378e implement more consistent InspectService that broadcasts Docker requests to all machines
* b73577e1a05113439f6058e0de328ebc4120094f implement subscription in corrosion client
* 5094149406d55c4119b03e9160fed64c9f22d3a5 improve error message for adding machine that already exists in the cluster
* 22bca1622628dd2038b89d5a4ba98652a6b51ea0 include alpha-numeric random string generation in secret package
* 95175d10c58d20c3dc55c02f1fd01067701d2f83 init Dockerfile to run machine in container
* 6eb373e4de91d9e24462f59f1ea25fdeb6760239 init InspectService method
* 667ecadd30286d400eeea634a678e47406abf468 init cluster store backed by corrosion
* c80de16ebe75501ad6ca289100e67c20dc3f984c init corrosion API client
* bd0e7f44b2cb1cf763551152ae5cc1e2b2d7e47a init corrosion admin client in machine
* 18ec186e81ebe2b179b198e3fb8322c1b173162e init landing page template
* 73785007cb7ffb66a30da02bb4752e20e59005ba init ucind (Uncloud in Docker) provisioner
* 7aa7a9fea199ec62ff9f1c7b4c1fe65894227b48 init ucind CLI commands
* c80c75e4e1a9bc225ce94fb1c21a2f2f915a47b9 install.sh: download corrosion binary from github release
* 498d35e866e89368460212b7ea6a47505a03f0e7 install.sh: install corrosion systemd service
* cf5818d9633a76ce471fd3ed732ed80a8e68a2a3 merge ucind cluster configuration to the user’s uncloud config
* 7d47d274a06280f5989fa1d7c27e90c8b071bcac minor
* bf10b21da50f17f6dcfb4d3214a6872440280394 minor comment
* 2835edadeb22b35b8a1b53cb492d4daebc8a10da minor landing wording update
* 21d55dc69aef3d240e82b9e9bcafe35ab5ddd766 minor refactor e2e cluster test
* 9c9e143dc8850b4483fa811350a45b962b5f412e minor rename
* deedbcb8a974490b9f3ba09212facf27180d897a minor validation for port
* 19d53c4f6724621b906a8d3144359fb6fce597d9 move Docker management from network controller to standalone DockerManager
* 24371eb401bbed7a2c807f7fec1e8fee74ee88e0 move Docker manager to docker package
* 5e7969750589783e6607bce2ce395714736a17c6 move RunService from CLI to the client, print output in the command
* 3ac166e6cb97cdd3e5b67d0afb27d117cf0aedf6 move common api types to api package
* 17b96a8744b7a7ef4f0e62ab34e8b1570df05cc4 move default uncloud socket to /run/uncloud/uncloud.sock
* a6ccd6055aaeaaef550e947ad8580ee89dcbc78e move fs and file ownership related funcs to fs package
* 5c1a3dd6949aaf3b59c8b346a0ae1d97f30b874c move generic docker types to top level docker package
* e8908ced9e109111ea94736a57954ecb3b26c14e parse --publish ports into spec
* 4088fdfd84d923e7ea7793eb727b715e4fd7230e print image pull progress when running a service
* bdd034d169300ba5fef03d521e0fc65acf4b8931 proper terminate all background processes in ucind containers
* 871fd8c035ec6c737d128458491ccd5f8ff84f34 proxy uncloud.sock socket to tcp port when running machine in container
* ca607ee82d4b536ebda1a01d3225133a14d13060 pull corrosion image if doesn't exist
* a40fd613e8c9d09d1aea3f7b95c4d254006c2f56 refactor RemoveService using InspectService
* 035f44896d94ec0632c9be4ad6f67cd099d9c95f refactor cluster network related constants
* 81cf5f56056eecdf6cc3400afd6070d2296aeabb refactor corrosion admin client to establish a connection on every command
* 06a4001080b4da49f9f04d3fb42cdc84a10d498c refactor corrosion client to accept custom http client, fix backoff for every request
* 2f7463ebe318b13eabf5e31a5535e97f04014001 refactor corrosion into interface service
* 68845e760778fbbfc624a8874d82774572412778 refactor corrosion service init for machine
* a4247b0097a5f6374054c84a68d846482a445a3c refactor grpc proxy backends to use machine management IP in metadata, including local responses
* 00dbbec62b127fe254804a365f58650821c94acc refactor grpc-proxy backends to enhance responses from local backends with metadata in one2many mode
* 6736839ccbbcc93d73d37e542a18c0bd6d72cc5c refactor machine ls command: move formatter logic to the command
* 2c62e1f6df27069e943c801ace38a1f4413fc82d refactor machine name generation using secrets package
* d922f7bfedd95f8c152ea535545aff6a3065ad7a reformat index.html
* 8fbf40600751dcddf51d2c7620b6bb8e1bdb6f85 relax relationship between corrosion service, init cluster with new store
* 417427b4d50b8e7264aef6e683e77f8357648fe0 remove duplicate ErrNotFound in client
* be2a26e626dee1cd34b766cf4acb4ad3266b3813 remove example from run command
* 1ec3376be51fe612667e13bcf36d35b8061338cc remove indentations from corrosion config
* b688742e5a62b3096dc5a91747bbaf12c857e47c remove the concept of cluster user (at least the current implementation)
* 8eea34d8dc8ea017adbf17a0e0cb675c83f27775 remove wrong TODO
* b3102480c033e3ccbd3743d53ec3e69a876ca805 rename site -> website
* d36429e6e578404e56747b1f8f1abe0a42caffe5 replace API server with transparent gRPC proxy that routes requests to local machine API
* c3d3a4242e4bc9cb472d8a74d5a826f88e2c2249 run corrosion in Docker as uncloud user, bind mount data dir
* 01a71b4b386fe8ca6da0cfffcb73dbd14d83f5c8 run each container in global service in parallel
* d31595820d2712eef1f5e9d81be2443b31736921 run service container in uncloud Docker network
* 5bf3cba2d66b57d14134b07ba8fccf1aaf460c4c sketch run command
* 840943b3b04f72a09d21395be82ad5f728cff74d split corrosion client subscription in a separate file
* 518eae9bd2e3c6d797dc6aef6ea487b1f2ac076f split corrosion queries in a separate file, make subscription more consistent with queries
* 3287691a314b75ce4ff2b805f982422f604bf814 start corrosion service even when machine is not initialised to be able to init cluster
* 77890b8a8a36f385ed815de2c04003e6f02a842a strip one proto layer in ListMachines method
* 54a8b09c9f72c2cf1bbfe0fdb5ea63f537bbb33f subscribe to Docker container events with backoff
* 6b80ef9db8473ee21d9a221853534b4eef447c19 subscribe to container changes in Caddyfile controller
* 4493bb3c271423d898fc6cb8ef86a4380bbd4790 support gRPC proxy to remote backends
* 5da1fc53879bffb805bdc8d50ba73af0d24d5d8d support providing TEST_CLUSTER_NAME for e2e tests
* b4c2ec3a2a0fd656288c12ba08bca6fab9326686 sync new and updated containers to cluster store, no deletion yet
* d5f0bca4a01156753bb9dcd3a749a4b0fba23342 ucind: call cleanup on exit as well, e.g. main process crash
* cd60d5178f29886c955bc64341694b0ec0a00476 ucind: create machines in containers with published API ports
* 48767125f4067138741db73904d3a41f085f3537 ucind: init cluster with created machines
* faf3378659937ac2fbaf81e5367f2117703627ba ucind: wait for machine to become ready through polling API
* 913fb2da0ceb2ec621813c9c01453a22273d8c6a update ListMachines to include machine state from Corrosion membership
* d02fc4d981f1e83a01ba61ab0e6903ccc0577234 update Makefile to remove built binary
* 91f870989a51c98a8d99e9d79e0320b186335d83 update README with basic Quick start
* 6daa1291b5bc8308f65db6b7450a1fe4bada9e99 update TODO
* 410f74425ca10ee3c164df9d19ecc78f21ecbda3 update dind machine image to run dockerd in background
* 6a4a7f2e6a510f7571dce6a7b0db046e5e11f1fc update e2e lifecycle test to test store reconciliation on every node
* f06dd865faa6d764abc67149ec137b0368608dc8 update existing WG peers instead of replacing to not interrupt and lose track of handshakes
* 6c197085d058ab7add760716d6b6d3edd41bb5ab update hero
* ba67b96734b76777c2d2f9eb655bde11c2b0a2b3 update install.sh to download uncloudd binary from the latest GitHub release
* 2e00632959e0e9cbc6384353fe4e6985d92227de update landing CSS
* fd3ad911361a490cc3b7fa544eff74a619eee9ac update network controller to reconfigure network peers on machine changes
* 537789902bac0ce92a54fa76edb8eddb4428dc78 update publish flag description with lb and host formats
* 21e70dd84668edb9a5353f192b3e377e831fb7e9 update quick start links on landing
* a54666a7ac832a232e170d77bcd80bec776a2224 update run command to run a stub service container on the connected node
* 3e9718f08c8d4ebeec9c58054b2e4459a4623057 update store schema when configuring corrosion, create machines table
* 411c3398ce577d8d32eb70ba5b80c4ecee46f985 upgrade to Go 1.23
* 61dc97228d0c484b07b193fdeb52c294e05dcc99 use ls command instead of list by default
* c70c8a6ec1044384cdaa0414df3722e2b96fb08d validate machine info before returning in ListMachine
* 8bddbec7bdeb9979e052a184cd9a05325f772f7b wait for test cluster to be ready (all machines up)
* 3a644771ca582b25e4c520e22d53a8be3c9f4d97 watch for peer endpoint changes and preserve in machine state
* 1c8e60aac058761318f50c8df83b75bba737886d workaround cgroup race in dind