Skip to main content

2024-09-29

Too much efficiency makes everything worse (2022)

  • The strong version of Goodhart's law suggests that over-optimizing a proxy measure can lead to worse outcomes in the actual goal, as seen in standardized testing and machine learning overfitting.
  • This concept is applicable across various fields, including politics, economics, and health, indicating the broad relevance of the phenomenon.
  • Mitigation strategies from machine learning, such as aligning proxy goals with desired outcomes, adding regularization penalties, injecting noise, and using early stopping, can help manage these issues.

Reactions

  • Over-optimization in machine learning and other fields can lead to negative outcomes, as suggested by ML researcher Jascha Sohl-Dickstein.
  • This concept aligns with Goodhart's law, which states that when a measure becomes a target, it ceases to be a good measure.
  • Examples of negative outcomes from over-optimization include COVID-19 supply chain disruptions and inefficiencies in healthcare and railways in Sweden, highlighting the need for systems to maintain some slack for robustness and adaptability.

How Discord stores trillions of messages (2023)

  • Discord initially used MongoDB for message storage but switched to Cassandra for better scalability and fault tolerance, which later led to performance and maintenance issues.
  • In 2022, Discord migrated from Cassandra to ScyllaDB, a more efficient, C++-based, Cassandra-compatible database, reducing nodes from 177 to 72 and significantly improving latency and performance.
  • The migration involved dual-writing new data and using a Rust-based migrator for historical data, resulting in fewer issues and better handling of increased traffic during major events like the World Cup.

Reactions

  • Discord transitioned from Cassandra to ScyllaDB to address performance issues, especially with deletes and garbage collection (GC).
  • ScyllaDB provides better compaction strategies and performance, despite still using tombstones for deletions.
  • Discord maintained their existing schema and partitioning strategy during the migration, emphasizing the importance of good default configurations.

SpaceX launches mission for 2 NASA astronauts who are stuck on the ISS

Reactions

  • SpaceX launched a mission to return two NASA astronauts from the ISS due to issues with Boeing's Starliner.
  • An anomaly occurred during the Falcon 9 rocket's second stage deorbit burn, prompting a pause in launches for investigation.
  • The astronauts will return using SpaceX's Crew Dragon, with new suits provided, sparking debate over whether this is a "rescue" mission or a routine crew rotation.

Building a robust frontend using progressive enhancement

  • Progressive enhancement starts with HTML, then adds CSS and JavaScript, ensuring basic functionality and accessibility for all users, including those with device or connectivity limitations.
  • JavaScript should enhance, not replace, HTML and CSS functionality; use feature detection, polyfills, and transpiling to ensure compatibility.
  • Avoid Single Page Applications (SPAs) as they can hinder accessibility and navigation; ensure your service remains functional despite potential CSS/JavaScript failures.

Reactions

  • The post discusses the benefits of building frontends using progressive enhancement, focusing on HTML and CSS with minimal JavaScript, as exemplified by gov.uk.
  • Many developers express frustration with the unnecessary complexity introduced by Single Page Applications (SPAs) and modern JavaScript frameworks, advocating for simpler, more maintainable solutions.
  • The conversation highlights a growing trend towards reconsidering simpler, HTML-based solutions, with tools like htmx gaining attention for their ability to reduce frontend complexity.

Notion's mid-life crisis

Reactions

  • Notion, initially popular for its customizable and versatile features, is facing criticism for becoming cluttered and less effective, similar to other overused tools like Jira.
  • Users express frustration with outdated and hard-to-find documents, suggesting that simpler tools like Google Docs might be more productive.
  • Despite its issues, Notion's database capabilities and flexibility still make it a strong contender for organizing and categorizing notes, though some users are considering alternatives like Obsidian.

Britain buys semiconductor factory for defence purposes

  • The UK government has acquired a semiconductor factory in Newton Aycliffe, County Durham, now named Octric Semiconductors UK, to support the defence supply chain and Armed Forces.
  • This facility is the only secure site in the UK capable of manufacturing gallium arsenide semiconductors, essential for military platforms like fighter jets, securing up to 100 skilled jobs.
  • The acquisition ensures the continued production of critical semiconductors for military applications, stabilizing the plant and enhancing the UK's defence capabilities and industrial capacity.

Reactions

  • The UK has purchased a semiconductor factory for defense purposes, highlighting the strategic importance of domestic semiconductor production.
  • The factory, previously owned by Coherent, specializes in gallium arsenide semiconductors, crucial for military applications like fighter jets and high-frequency radars.
  • This acquisition secures up to 100 skilled jobs and ensures the UK maintains a critical part of its defense infrastructure, especially as global semiconductor demand rises.

Floating megabomb heaves to near the English coast

  • The MV Ruby, a Maltese-registered ship carrying 20,000 tons of explosive ammonium nitrate, is damaged and seeking a port off the coast of Kent, UK, posing a significant security concern.
  • The ship has been denied entry to several European ports, including Norway and Lithuania, due to the dangerous nature of its cargo, which could cause devastation comparable to a third of the Hiroshima bomb.
  • The incident highlights Russia's hybrid warfare tactics and underscores the need for robust intelligence-sharing, surveillance, and maritime security protocols to manage such threats effectively.

Reactions

  • A damaged ship carrying 20,000 tons of ammonium nitrate, a highly explosive material, is nearing the English coast, causing significant safety concerns.
  • The ship has been denied entry to multiple European ports and is suspected of being used by Russia to harass NATO countries, raising geopolitical tensions.
  • Authorities are on high alert due to the potential threat and Russia's history of low-level hostilities, questioning the ship's intentions as it travels from Norway to Lithuania and now near the UK.

Some Go web dev notes

  • Go 1.22 introduces improved routing support in the standard library, simplifying route handling and enhancing security with middleware.
  • sqlc tool generates Go code from SQL queries, reducing boilerplate and streamlining database interactions.
  • Setting a GC memory limit in Go 1.19 helps prevent out-of-memory (OOM) kills in low-memory environments, enhancing application stability.

Reactions

  • Go's simplicity and stability are appreciated by developers, making it easy to resume projects after long breaks.
  • Initial setup can be challenging due to the need to select various libraries for logging, database drivers, etc.
  • The lack of a dominant framework in Go is seen as both a limitation and a benefit, offering flexibility with standard libraries.

Notes on the Crystal Language

  • Crystal, a language started in 2014, aims to be a strongly-typed Ruby with type inference and compiles to native executables via LLVM.
  • It features a decentralized ecosystem of libraries, optional type annotations, and a type system similar to TypeScript’s, supporting generics and other object-oriented features.
  • Despite some issues with tempfile handling and directory iteration, Crystal offers many useful built-in functions and libraries, making it a solid choice for those looking for a dynamic-feeling static type system.

Reactions

  • Crystal language is being discussed for its potential and current limitations, with users sharing mixed experiences regarding its reliability and maturity.
  • Some users highlight Crystal's high performance and productivity for backend API servers and background tasks, while others point out issues like slow garbage collection (GC) and bugs.
  • The conversation includes comparisons with other languages like Ruby, Clojure, and Rust, and mentions Crystal's use of LLVM for optimized code generation, making it comparable to Go, Rust, or C in performance.

Web components are okay

  • Nolan Lawson addresses the debate on web components, sparked by Ryan Carniato's post "Web Components Are Not the Future" and Cory LaViska's counter "Web Components Are Not the Future — They’re the Present."
  • Lawson, experienced with web components, acknowledges their performance overhead but highlights trade-offs such as maintainability, security, usability, and accessibility.
  • He concludes that web components, despite their limitations, offer unique creative opportunities and are one of many tools available in the diverse web development landscape.

Reactions

  • The discussion centers around the complexity and usability of modern front-end frameworks versus web components, with some developers expressing frustration over the "magic" in frameworks like React.
  • Web components are highlighted for their simplicity and closer alignment with web standards, offering features like Shadow DOM for isolation without the need for complex frameworks.
  • A developer shared their positive experience transitioning from Vue to web components, significantly reducing dependencies and maintenance overhead, which is particularly beneficial for projects with limited resources.

htmgo - build simple and scalable systems with golang + htmx

  • A developer is working on a minimal stack combining Golang and htmx to enhance features like components and improve syntax for complex JavaScript.
  • The project aims to provide a lighter alternative to heavy frameworks while addressing the low-level nature of htmx.
  • This initiative could be beneficial for other developers seeking a streamlined and efficient development stack.

Reactions

  • A developer introduced "htmgo," a minimal stack combining Golang and HTMX, aimed at simplifying the creation of reusable components and improving syntax.
  • The project has sparked discussions among developers about the merits of integrating backend languages with HTMX and the benefits of using Golang for lightweight web development.
  • The community is comparing "htmgo" with other tools and frameworks like Hotwire, Gomponents, and various templating engines, highlighting a growing interest in efficient, server-side web development solutions.

Flexible RISC-V Processor: Could Cost Less Than a Dollar

  • A new 6-mW open-source plastic chip based on the RISC-V architecture can perform machine learning tasks while being flexible enough to bend around a pencil.
  • The chip loses only about 4 percent of its performance when bent, showcasing its robustness and potential for flexible electronics applications.
  • This development highlights significant advancements in the field of flexible semiconductors and open-source hardware.

Reactions

  • A recent IEEE article highlights a flexible RISC-V processor that could cost less than a dollar, based on the SERV design.
  • Pragmatic Semiconductor claims they can produce these chips within 48 hours, significantly faster than traditional silicon fabrication processes.
  • Although the processor runs at 60 kHz and consumes 6 milliwatts of power, it could be valuable for niche applications such as e-textiles and flexible electronics.

NASA is selling a brand-new Moon rover: Never used, one previous owner

  • NASA aims to return humans to the Moon by the end of the decade, starting with probes to the lunar south pole to search for ice, which holds scientific and practical importance.
  • Recent advancements include immune therapies for asthma, heart disease, and aging, new tech for detecting water pipeline leaks, and the discovery of ancient cheese in China.
  • Innovations in AI and electric-car battery recycling are also noteworthy, showcasing ongoing progress in various tech fields.

Reactions

  • NASA is selling a new, unused Moon rover, sparking discussions about its value and historical context on a forum.
  • Users debate its worth, share technical details, and humorously comment on the topic, while some express concerns about off-topic content.
  • The conversation also highlights potential corruption in NASA's contractor bidding process and the challenges of building during COVID.

Feldera Incremental Compute Engine

  • Feldera is a high-performance query engine designed for incremental computation, allowing continuous processing of changes without recomputing older data.
  • It supports full SQL syntax and connects to various data sources, enabling users to manage pipelines and inspect results in real-time, even with datasets larger than RAM.
  • Feldera can be started using Docker Compose or from source with dependencies like Rust, Java, Maven, and Typescript, and offers a web console for user interaction.

Reactions

  • Feldera Incremental Compute Engine is an open-source tool designed for incremental computation, reducing stream processing times significantly.
  • It is based on Z-Sets, a generalization of relational algebra, allowing many SQL operations to be implemented incrementally.
  • Feldera supports a wide range of SQL operators, automatic garbage collection, and can index data on storage, making it suitable for sophisticated SQL programs and historical data warehouses.

Text2CAD Generating Sequential CAD Designs from Text Prompts

  • Text2CAD is the first AI framework designed to generate parametric CAD (Computer-Aided Design) models from multi-level textual descriptions, making it a significant advancement in the field.
  • The framework includes a novel data annotation pipeline using open-source LLMs (Large Language Models) and VLMs (Vision-Language Models) to annotate the DeepCAD dataset with varied text prompts.
  • Text2CAD Transformer employs a pretrained BeRT Encoder and Adaptive layer to transform natural language descriptions into 3D CAD models, showcasing its capability through both qualitative and quantitative results.

Reactions

  • Text2CAD is a tool that generates CAD (Computer-Aided Design) models from text prompts, aiming to simplify the design process for users unfamiliar with traditional CAD software.
  • The tool is particularly beneficial for users who may not frequently design objects, as it reduces the need for extensive CAD training, though it may not be as efficient for complex designs.
  • The discussion highlights the potential and limitations of using natural language to create precise 3D models, with some experts suggesting that traditional CAD workflows are still more effective for detailed and accurate designs.