Is Apache Spark too costly? Amazon Principal Engineer Patrick Ames tackled this question during an interview with The New Stack Makers, sharing insights into transitioning from Spark to Ray for managing large-scale data. Ames, described as a "go-to" engineer for exabyte-scale projects, emphasized a goal-driven approach to solving complex engineering problems, from simplifying daily chores to optimizing software solutions.

In this New Stack Makers, Codiac aims to simplify app deployment on Kubernetes by offering a unified interface that minimizes complexity. Traditionally, Kubernetes is powerful but challenging for teams due to its intricate configurations and extensive manual coding. Co-founded by Ben Ghazi and Mark Freydl, Codiac provides engineers with infrastructure on demand, container management, and advanced software development life cycle (SDLC) tools, making Kubernetes more accessible.

Valkey, an open-source fork of Redis launched in March, introduced its multithreaded Version 8.0 in September, now available through AWS ElastiCache. AWS's Kyle Davis explains that Valkey was developed after Redis changed to a restrictive license, drawing contributors from companies like AWS, Google, Alibaba, and Oracle. Notably, some contributors emerged independently, including a significant contributor from Vietnam. Version 8.0 differentiates itself from Redis by leveraging multithreaded...

In this episode of the New Stack Makers, Deb Nicholson, executive director of the Python Software Foundation, attributes Python’s popularity to its minimal syntactical complexity, which appeals to beginners and seasoned developers alike. Python allows flexibility for those exploring coding without a specific focus, unlike purpose-built languages. Since her leadership began in 2022, Nicholson has overseen the foundation’s role in managing Python’s fiscal and operational needs, including the...

Platform engineering will be a key focus at KubeCon this year, with a special emphasis on AI platforms. Priyanka Sharma, executive director of the Linux Foundation, highlighted the convergence of platform engineering and AI during an interview on The New Stack Makers with Adobe’s Joseph Sandoval. KubeCon will feature talks from experts like Chen Goldberg of CoreWeave and Aparna Sinha of CapitalOne, showcasing how AI workloads will transform platform operations.

Rohit Choudhary, co-founder and CEO of Acceldata, placed an early bet on data observability, which has proven prescient. In a New Stack Makers podcast episode, Choudhary discussed three key insights that shaped his vision: First, the exponential growth of data in enterprises, further amplified by generative AI and large language models. Second, the rise of a multicloud and multitechnology environment, with a majority of companies adopting hybrid or multiple cloud strategies. Third, a shortage...

Rust has maintained its place among the top 15 programming languages and has been the most admired language for nine consecutive years. In a New Stack Makers podcast, Joel Marcey, director of technology at the Rust Foundation, discussed the language's growing importance, including initiatives to improve its security, performance, and adoption in various domains. While Rust is widely used in systems and backend programming, it’s also gaining traction in embedded systems, safety-critical...

In this New Stack Makers episode, Ashley Williams, founder and CEO of axo, highlights how the software world depends on open-source code, which is largely maintained by unpaid volunteers. She likens this to a CVS relying on volunteer-run shipping companies, pointing out how unsettling that might be for customers. The conversation focuses on open-source maintainers’ reluctance to be seen as "suppliers" of software, an idea explored in a 2022 blog post by Thomas Depierre. Many maintainers...

In this New Stack Makers podcast, Xun Wang, CTO of Bloomreach, brings insights from his time at Nvidia, particularly lessons from its founder, Jensen Huang, to his current role in e-commerce personalization. Wang emphasizes structuring organizations to reflect the architecture of the products they build, applying a hands-on, detail-oriented approach that encourages deep understanding of engineering challenges.

Code reviews can be highly beneficial but tricky to execute well due to the human factors involved, says Adrienne Braganza Tacke, author of *Looks Good to Me: Actionable Advice for Constructive Code Review.* In a recent conversation with *The New Stack*, Tacke identified three challenges teams must address for successful code reviews: ambiguity, subjectivity, and ego.

In the New Stack Makers episode, Adi Polak, Director, Advocacy and Developer Experience Engineering at Confluent discusses the operational and analytical estates in data infrastructure. The operational estate focuses on fast, low-latency event-driven applications, while the analytical estate handles long-running data crunching tasks. Challenges arise due to the "schema evolution" from upstream operational changes impacting downstream analytics, creating complexity for developers.

Bob Wise, CEO of Heroku, discussed the impact of generative AI (GenAI) coding tools on software development in a recent episode of The New Stack Makers. He compared the rise of these tools to adding an "infinite number of interns" to development teams, noting that while they accelerate code writing, they don't yet simplify testing, deployment, or production operations. Wise likened this to the early days of Kubernetes, which focused on improving operations rather than the frontend experience....

After the XZ Utils backdoor vulnerability was uncovered in March, the OpenJS Foundation saw a surge in inquiries from potential open source JavaScript contributors. Robin Ginn, executive director of the foundation, noted that volunteer-led JavaScript communities often face challenges in managing these contributions. The discovery that a single contributor, "Jia Tan," planted the backdoor heightened vigilance, especially when new contributors requested admin privileges. Ginn emphasized that...

Paige Bailey, who began coding at age 9 in rural Texas, now leads the GenAI developer experience at Google. In a conversation with Chris Pirillo on The New Stack Makers, Bailey reflected on the evolving role of software development in the era of generative AI. While she once urged her nieces and nephews to pursue computer science degrees, Bailey now believes that critical thinking and problem-solving may be more crucial for future tech careers.

Anne Currie, a leading expert in sustainable tech and part of the Green Software Foundation, discusses practical steps for building resilient, sustainable software in an episode of The New Stack Makers. With 30 years of experience, Currie co-authored Building Green Software, emphasizing the tech industry's role in the energy transition. She highlights the complexity of adapting technology to renewable energy, involving extensive research and debunking misinformation. Currie discusses the...

In an era marked by complexity, the golden path is essential for software architects, asserts James Watters, senior director of R&D at VMware Tanzu, Broadcom. This approach, emphasizing fewer application patterns, simplifies life for security personnel, developers, and infrastructure teams. VMware defines the golden path as streamlining software development, crucial in today's economic climate. Watters highlights this in the Broadcom report: State of Cloud Native App Platforms 2024,...

Maintaining and ensuring the success of a microservice-based system can be challenging. Sarah Wells, a seasoned tech consultant with over 20 years of experience, offers valuable insights in her book "Enabling Microservices Success" and a discussion on The New Stack Makers podcast. Drawing from her tenure at the Financial Times (FT), Wells illustrates how transitioning to microservices and adopting DevOps and SRE practices enabled FT to accelerate software releases from 12 annually to over...

In August 2023, the open source community rallied to create OpenTofu, an alternative to Terraform, after HashiCorp, now owned by IBM, adopted a restrictive Business Source License for Terraform. Ohad Maislish, co-founder and CEO of env0, explained on The New Stack Makers how this move sparked the initiative. A few hours after HashiCorp's license change, Maislish secured the domain opentf.org and began developing the new project, eventually named OpenTofu, which was donated to The Linux...

In the early days, the internet was a decentralized space created by enthusiasts. However, it has since transformed into a centralized, commerce-driven entity dominated by a few major players. The promise of the fediverse, a decentralized social networking concept, offers a refreshing alternative. Evan Prodromou, OpenEarth Foundation's director of open technology, has been advocating for decentralized social networks since 2008, starting with his creation, Identi.ca. Unlike Twitter,...

In a recent episode of The New Stack Makers, recorded at the Open Source Summit North America, Matt Hartley, Linux support lead at Framework, discusses the importance of the "right to repair" movement. This initiative seeks to allow consumers to repair and upgrade their own electronic devices, countering the trend of disposable electronics that contribute to environmental damage. Framework, a company offering modular and customizable laptops, embodies this philosophy by enabling users to...

Blockchain technology continues to drive innovation despite declining hype, with Distributed Ledgers (DLTs) offering secure, decentralized digital asset transactions. In an On the Road episode of The New Stack Makers recorded at Open Source Summit North America, Andrew Aitken of Hedera and Dr. Leemon Baird of Swirlds Labs discussed DLTs with Alex Williams.

The Linux xz utils backdoor exploit, discussed in an interview at the Open Source Summit 2024 on The New Stack Makers with John Kjell, director of open source at TestifySec, highlights critical vulnerabilities in the open-source ecosystem. This exploit involved a maintainer of the Linux xz utils project adding malicious code to a new release, discovered by a Microsoft engineer. This breach demonstrates the high trust placed in maintainers and how this trust can be exploited. Kjell explains...

Suman Debnath, principal developer advocate for machine learning at Amazon Web Services, emphasized the advantages of using Python in machine learning during a New Stack Makers episode recorded at PyCon US. He noted Python's ease of use and its foundational role in the data science ecosystem as key reasons for its popularity. However, Debnath highlighted that building generative AI applications doesn't necessarily require deep data science expertise or Python.

Nathan Peck, a senior developer advocate for generative AI at Amazon Web Services (AWS), shares his experiences working with Python in a recent episode of The New Stack Makers, recorded at PyCon US. Although not a Python expert, Peck frequently deals with Python scripts in his role, often assisting colleagues in running scripts as cron jobs. He highlights the challenge of being a T-shaped developer, possessing broad knowledge across multiple languages and frameworks but deep expertise in only...