Issue# 112
Scaling Postgres, Unison a programming language, Open source not open contribution, High price of mistrust, Fast flexible paxos ...
“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.”
– Martin Fowler
Posts
Software development topics I've changed my mind on after 6 years in the industry
-Things I've changed my mind on
-Opinions I've picked up along the way
-Old opinions unchanged . - #chriskiehl
Open-source, not open-contribution
I do not wish to come off as anything but welcoming, however, I've made the decision to keep this project closed to contributions for my own mental health and long term viability of the project. - #github
Developers: Start Using Open MCT
Open MCT is a next-generation mission operations data visualization framework. Web-based, for desktop and mobile. - #nasa #github
Cracks are showing in Enterprise Open Source's foundations
- Red Hat effectively killed CentOS
- Elastic effectively killed Elasticsearch
- #jeffgeerling
POSTGRES SCALING ADVICE FOR 2021
-PostgreSQL and MySQL – brothers from another mother
-All distributed systems are inherently complex, and difficult to get right
-What does plain PostgreSQL offer?
-Tips to be prepared for scaling - #cybertec #postgresql
When we can’t trust each other, nothing works. As we participate in our communities less and less, we find it harder to feel other people are trustworthy. But if we can bring back a sense of trust in the people around us, the rewards are incredible. - #fs #blog
So, You Want to Scan a Billion Files in a Day?
How many files do you have on your computer? How much total space do they take up? Can you, armed with only the file name, find a specific file that you worked on 3 years ago? If your disk is getting full, which files would you delete or move first, in order to free up space - #medium
Fast Paxos’ intersection requirements can be safely relaxed, reducing to just one additional intersection requirement between phase-1 quorums and any pair of fast round phase-2 quorums. We thus find that the quorums used with Fast Paxos are larger than necessary, allowing alternative quorum systems to obtain new tradeoffs between performance and fault-tolerance.
- #srcf #HeidiHoward
Identify requests from clients uniquely so they can ignore duplicate requests when client retries
- #martinfowler #UnmeshJoshi
Introduction to Instrumentation and Observability in Distributed Systems
A distributed system is a system whose components/services are located on different networked hosts, which coordinate their actions by passing messages to one another. The components interact with one another to achieve a common goal. - #medium
Don't Get Stuck in the "Con" Game
- #pathelland #substack
Handling Complexity: Using Sagas to Provide Transactional Support for Distributed Systems
Distributed systems, especially with the use of the Microservice Architecture, are the new normal. Long gone are the days where we could take for granted the atomic nature of using a single relational database and monolithic system to provide our users with the functionality they expect. - #medium
Applying The Universal Scalability Law to Distributed Systems
After demonstrating that both Amdahl's law and the Universal Scalability law (USL) are fundamental, in that they can be derived from queueing theory, the USL is used to quantify the scalability of a wide variety of distributed systems, including Memcache, Hadoop, Zookeeper, AWS cloud, and Avalanche DLT. Contact is also made with some well-known concepts in modern distributed systems, viz., Hasse diagrams, the CAP theorem, eventual consistency and Paxos coordination. - #speakerdeck
Scaling Kubernetes to 7,500 Nodes
We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models. Scaling a single Kubernetes cluster to this size is rarely done and requires some special care, but the upside is a simple infrastructure that allows our machine learning research teams to move faster and scale up without changing their code. - #openai
In November 2020 we began a large migration to upgrade our PostgreSQL cluster from version 9.6 to 12.4. In this post I’ll give an overview of our architecture at Coffee Meets Bagel, walk through the steps we took to perform this upgrade with less than 30 minutes of cumulative downtime, and share some lessons learned along the way. - #tech #coffeemeetsbagel
Podcast
In this episode Rich speaks with André Henry from Venmo. Topics include: How André’s mom was the first programmer he knew, working at ISPs in the 1990s, why infrastructure engineers love abstractions, Kubernetes is not a PaaS, how Docker nailed developer experience, and what may be coming for distributed computing. - #share #transistor
Research
Security, Fault Tolerance, and Communication Complexity in Distributed Systems
we address the problem of computing a general function of several private inputs distributed among the processors of a network, while ensuring the correctness of the results and the privacy of the inputs, despite accidental or malicious faults in the system. - #DonaldRozinakBeaver
Video
PHASE: Rúnar Bjarnason on Unison
Unison is an open source functional programming language based on a simple idea with big implications: code is content-addressed and immutable.