An interview with Liran Tal, Director of Developer Advocacy at Snyk and a GitHub Star

We recently had the pleasure of having a fantastic conversation with Liran Tal, who holds the position of Director of Developer Advocacy at Snyk and is also a GitHub Star. Get ready to delve into a captivating interview with Liran, along with our CTO Eran Yahav, and VP Product Yossi Salomon from Tabnine. During this interview, we discussed topics such as security vulnerabilities, generative AI for coding in open source, and the intriguing places that saying “yes” can take you. 

Buckle up and enjoy!

How did you become such an expert in the world of open source? 

I think it’s mainly about loving technology, trying out many new things, and not being afraid to dive into various new projects and opportunities. One of my secrets is that I almost always say yes to things – which can be good and bad. Honestly, it’s just a matter of being open and optimistic. With open source, it goes back to Linux. I started with Linux installations on all kinds of old computers, then ran BBSs in the days before the internet became mainstream in my country, and then I got into the KernelJanitors Linux mailing lists, etc.

So your advice would be to say yes to everything? Seems like a pretty risky approach, no? 

I mean, it’s worked out pretty well for me. I know more or less how to manage my time. 

My first exposure to the tech world was in a company that was a systems integrator, offering Linux services and implementing VoIP Asterisk systems. I was more in a SysAdmin kind of role, and I realized I wanted to get deeper into programming. So I went to several interviews, one of which was an absolute disaster – but there was one company that decided to give me a chance. 

One of the first tasks I was given in my role there was to write a state machine in C++. I told the R&D manager that I’ve never written C++ before, and he handed me a book on C++ and told me to get to it. It took me a couple of weeks to read the book, and I started writing C++. That’s just an example of how important it is to say yes to opportunities. This is especially important in open-source, where there aren’t many set rules, and the guidelines are more oral, like “be a good citizen” or “don’t be an a******”. The most challenging thing about the open-source community is trying to move things forward since most contributors are working part-time on these projects. So if someone is willing to contribute, whether it’s coding, documentation, or anything else, they’re usually warmly welcomed.

Have there been any situations where you felt like you bit off more than you could chew? 

For me, it’s all a learning experience. Even in my current role as a Developer Advocate at Snyk – I had no real idea what the role entailed. Before this role, I was either a development team leader or a programmer. I’ve done public speaking at conferences around the world for many years talking about open-source, Node.js, JavaScript, etc, but it was more of a hobby. Then someone from Snyk approached me and asked if I wanted to do it full-time. To be honest, I contemplated it for a couple of months – it was a pretty big career pivot that scared me – but I finally decided to go for it.

What are your day-to-day responsibilities in the role?

I’ve been very lucky in this role – I’m very proactive and I have lots of ideas, and this role gives me the opportunity to actually try them out. There are everyday responsibilities, including education, content creation, etc. For example, I also write command-line applications, so I created a CLI tool that scans all sorts of websites for security issues, called is-website-vulnerable. And it’s totally connected with the whole world of DevOps security. It didn’t even have a formal Jira story – it was a weekend side-project that suddenly exploded with over a thousand stars on GitHub. Lots of people started contributing to it and making such a CLI that puts everything in JSON, and then someone wrote a GitHub Action, and someone else connected it to his pipeline. That’s just one of many examples, especially when it comes to DevRel (developer relations), because it touches on so many other areas – product, development, end-user developers, education….

Over the last year, for example, I’ve done a lot of security research. I’m not a security researcher or analyst by trade, but I was fascinated by it, and ended up finding and disclosing a lot of security vulnerabilities in different open-source libraries.

One of the main concerns around generative AI is security. What, from your POV, is the right approach to introducing AI to enterprise R&D teams, and integrating it with secure coding paradigms? 

The concern here is real and tangible. On the other hand, I feel like things are a bit…all over the place at the moment. For example, I use ChatGPT, and something I’ve noticed (with me as well) is there’s this kind of “I want to get it done fast” mentality amongst developers. For example, I tell ChatGPT what I need, it gives it to me, and I copy/paste and run it. But that’s not really that new, you know? Before ChatGPT, we did the same thing. We’d ask Google, it directed us to Stack Overflow, we found an answer, saw that a million people upvoted it, and we copy-pasted it. So it’s not like things have changed that much for developers.

But on Stack Overflow, you’ve got some indication regarding quality and trustworthiness: Upvotes, downvotes, stars, downloads, etc. ChatGPT speaks with absolute confidence, whether it’s speaking truth or nonsense.

The concern you raise is valid and would be even more pressing if ChatGPT was within the IDE. Currently, you have to copy-paste from the web chat into the IDE, which still involves some friction. It’s small, but still enough to make it impractical, so a developer probably won’t do it constantly. Once it’s within the IDE, it becomes a much larger source of concern. I’ll put a disclaimer in here – I’m not a data scientist, and I don’t pretend to be. The issue is around how the models are trained. The ChatGPT model that I’m familiar with generates code based on the code it’s been exposed to, which obviously includes bad code as well, with lots of security vulnerabilities. So the source of the data is definitely another concern. How was the model trained? How robust is the data? Is it secure, high-quality, high-performance, fast, efficient in terms of memory use, etc.? Has the function that ChatGPT generated for me been checked, secure-coded, tested, etc? We know that the answer to all of these is probably no. 

So what are our expectations? It’s probably no worse than code written by a developer, but maybe our expectations are higher because supposedly a machine can do better.

I think that there’s an issue of awareness. Just this morning I arrived back from London. I was at a conference called CityJS, and I gave a lecture on “Path Traversal attacks”, teaching developers about a specific vulnerability that occurs in very popular open-source libraries that have been downloaded millions of times as well as impacted the Node.js runtime. The evening before my talk I was reviewing my slides and decided to open my lecture with a story. So I asked ChatGPT to create a file upload route in the Fastify web framework Node.js application, without any dependencies. The first code it generated looked really good. Functionally it almost worked. There was something small missing from the code – a method – which I fixed. It then regenerated the code for me. In both versions, there was a vulnerability that I noticed, but didn’t comment on, because I wanted to see how ChatGPT would handle it. In terms of the functionality, it now worked, but it still had a security vulnerability – basically, the code didn’t save the file in the correct way. So I opened my lecture by showing how commonplace these types of vulnerabilities are, even on code generated by ChatGPT – and I didn’t have to work hard to find an example at all. 

So how do we educate developers about the risks? 

Well, using common sense and discretion is a big part of it. The world of AI is here to help us work faster, but we still need to use our own judgment to steer it in the right direction. I was chatting with a friend who was consulting for a company and they showed him a bug generated by ChatGPT that he recognized from Stack Overflow – he’d seen the same bug in three different projects where developers had copied the same code, and ChatGPT had obviously trained on it as well. This highlights the importance of education. While augmented development can increase productivity, for critical tasks, we need to provide the machine with more detailed instructions, background, and context. We may need to use code repositories with low vulnerabilities or increase supervised learning for the models. It’s about using our judgment to direct the machine toward what we need.

In case you’re an enterprise looking to incorporate AI into your software development lifecycle, Tabnine is an exceptional option.

By utilizing Tabnine Enterprise, you’ll have the opportunity to benefit from contextual code suggestions that can boost your productivity by streamlining repetitive coding tasks and producing high-quality, industry-standard code. Tabnine code suggestions are based on Large Language Models that are exclusively trained on credible open-source licenses with permissive licensing.

What do you think about the idea of showing the developer how, for instance, a snippet of code is similar to a snippet on Stack Overflow? So for the example you mentioned, where code generated from ChatGPT had the same bug seen several times before in Stack Overflow, maybe something like that could be a help? 

That could be an interesting direction. Let’s try to apply this thought process to other things that are already happening in the ecosystem. For example, open-source libraries. Let’s say a developer wants to implement a certain functionality. They’d search for it in the libraries, and would probably find quite a few components to choose from, since the ecosystem is very mature at this point. In reality, I think they’d probably look at the code, see if there are tests, how many stars it has, how many downloads and contributors, and use it. So even before the developer starts writing the code, on the component level, there often isn’t the awareness that you need to go in and make sure that the code looks like it should. 

So what are the best practices? What do you suggest?

Well, for instance, at Snyk, we built a tool called Snyk Advisor, which examines four perspectives: Popularity, security, community, and maintenance, and then it provides a score for all 4. So for instance, let’s say there’s a package without a commit or release for over a year – that’s a flag that needs to be raised. The code might be amazing, with cross-platform testing and everything, but if it’s not maintained and has a vulnerability or even a bug, there’s no one to fix it on the spot, and you’ll end up stuck.  

There’s another cheat that I don’t often speak about. Node.js’s Canary in the Goldmine (CITGM) is a technology that pulls down modules from npm and tests them to see if they’ll break when Node.js updates its versioning.

Another tip is to look at prolific maintainers, like Matteo Collina and others. Look at what they wrote or what they choose to use. That goes to reputation, so if you know that a certain developer chose to use a specific package, they probably already did their due diligence, so you can use it. 

But say, for instance, the packages that make it into the Node test suite, they’re probably within the “trusted compute base,” right? 

There could be packages that have been downloaded 5 million times but could be terrible and break things. The CI is there to ensure that nothing in the ecosystem is broken, but it’s not an assurance that a package is good and should be used. So it works well as a basic filter, but it’s not enough to ensure the quality of the package.  

Let’s assume that in the future, code will be generated by tools like Tabnine, which are already in the IDE, and the volume of generated code (or copy-pasted) will be significantly larger than what we’re used to today. How will that affect the ecosystem, especially in the CICD? Does it transfer more responsibility to the gatekeepers? Obviously, developers will still be responsible for their code, but does it potentially transfer more responsibility to someone who’s supposed to catch issues further down in the SDLC funnel? As the manager of an R&D organization that adopts AI code generation, which processes and tools should I put in place to ensure that my code remains at the same quality as in the past? On the one hand, we’re creating systems that are far more accelerated. On the other hand, there’s a higher risk. So what can we do to ensure that production issues are prevented?

In the book by the Phoenix Project and the Accelerate report, as well as Puppet, which publishes a yearly “State of DevOps” report, they all talk about DevOps and link it to security. I think that the more mature an organization is, for example, when you’re already in the CICD phase with lots of feature flags, you’re very advanced in terms of technology, and you can fail fast, it’s much easier to integrate and use lots of different tools, for security and more. So maybe the controls won’t change much, but they’ll get sharper. 

As code is being generated faster with tools like Tabnine, there needs to be a greater emphasis on code review to catch any issues. Even with all the other security-related DevOps processes like dynamic and static application testing, someone needs to review the code to ensure it’s correct and working, including edge cases. Code review tools and processes aren’t meant to distinguish between good or bad coders, but to identify and catch specific issues. It’s particularly important for less mature companies to have a defined code review process because generating lots of code doesn’t necessarily mean it’s good code, and mistakes and vulnerabilities can easily slip through without proper testing and review.

In other words, the bottleneck in the dev process could move from writing code to code review or maybe testing, while the throughput remains the same.

Yes. If I were to give a parallel to the world of dependencies. If I go back a couple of “generations”, during the time when everything was embedded, you’d open your editor, create a new file project, with a blank page, and start writing. Today, you want to create a project, let’s say it’s Java, you’ve got Spring Starter Project or Spring Boot or whatever. Your default is to enter, like, twenty dependencies. And with JavaScript, it becomes a bit larger. So the default is that you’ve got lots and lots of code, and what I’m writing is maybe 10% business logic or something like that. So this world has really accelerated in a dramatic way.