The field of generative AI, featuring models like ChatGPT, is rapidly expanding in the realm of software development, potentially changing the way we tackle coding and problem-solving.
Tabnine recently held a Twitter space discussion on generative AI, with CTO Eran Yahav and moderation by Lisa Zigel. The topics covered included the current status of generative AI in software development, the emergence of models like ChatGPT, potential future uses, and how AI gives developers superpowers!
Other subjects discussed included the definition of software engineering, AI in the different stages of the SDLC, and the important role that AI will place in code review and testing.
Eran Yahav, the head of technology at Tabnine and a computer science professor at the Technion in Israel, is an expert in areas such as program synthesis, machine learning in code, program analysis, and verification. He’s recently spoken about the major effects of AI in software development and automating processes. To learn more about incorporating AI into your software development team, visit Tabnine Enterprise.
When a layperson imagines quality control, they usually imagine end-to-end testing. Approach the software as a user would and execute the same journey they would. The idea makes sense; If a tester can get through your software end-to-end, it works.
Well, it’s not that simple. Users make errors, get confused, enter insufficient data, and try to use the software for things it was not meant to. End-to-end testing attempts to map all those different scenarios and execute them before every software update to ensure the software still behaves as expected under regular use.
But even if you make a complete roadmap of user journeys through your software, end-to-end testing still has the potential to cause more problems than help find solutions. This article explores the common pitfalls of end-to-end testing and how you can leverage it better.
You can perform end-to-end testing using automated tools, manually, and sometimes via APIs. You may test a simple grocery list app by performing everyday tasks such as adding, removing, deleting, and checking list items. If the app is connected to a server, then the server can be tested using API calls performing those tasks, making the testing an automated process.
End-to-end testing aims to ensure that users can complete specific tasks in your application. It is done by running scenarios and then identifying failure points so the development team can fix them. Unfortunately, that is easier said than done.
End-to-end testing is often a necessity. You must see that the user processes are performing as expected and that no significant and obvious faults arise. But too high a focus on end-to-end testing is not ideal and can lead to slow development cycles and increased costs.
End-to-end testing naturally takes a long time. It must be thorough and slow, even if you review code using automation testing, scripting, or APIs. When you commit an update to the software, running a complete array of end-to-end tests is impractical due to the slow execution. Therefore end-to-end testing is reserved for the end of the development cycle.
But keeping testing until the end of the development cycle means that developers get slow feedback on their changes, and if you find an issue at this stage, you have to fix it late. To put an even more significant strain on the problem, any fix can cause other areas of the code to break, meaning you must restart end-to-end testing after the fixes are applied. This feedback loop could be faster and more efficient.
There are several ways to improve the efficiency of end-to-end testing:
To keep end-to-end testing as a failsafe, you should employ other testing methods, such as unit testing. Another option would be to use an intelligent auto-completion tool that can reduce bugs in your code. AI-trained tools like Tabnine or Copilot can, for instance, auto-complete your lines of code based on your best practices, style and patterns, to ensure consistency and accuracy. AI can also automatically generate fixes for bugs, making your testing process much easier.
Google reports about 1.5% of tests failing flakily, meaning tests that fail and pass with the same code. When those tests are unit tests, you can identify them as flaky by running them multiple times and then removing the flaky tests. With end-to-end testing, this would be much more difficult to detect.
End-to-end testing covers a lot of ground because the user interacts with many modules during a user journey. It is easy to see that if one portion of a long series of actions is flaky, the entire test becomes flaky. And if every step has a 1.5% chance of being unreliable, that can add up, resulting in end-to-end testing having a very high chance of being flaky themselves, much higher than 1.5%.
End-to-end testing run in a single environment is largely reproducible and consistent (see flaky testing), but most developers use distributed systems these days. Testing in distributed systems poses many challenges, especially for end-to-end testing, as the same user journey can produce vastly different results in different scenarios.
Teams testing on various platforms can have miscommunication issues and friction between them. To ensure end-to-end testing goes smoothly, you must synchronize technologies, data, and information processed between the teams. Using cloud and standard test runtime tools like WebLOAD and LoadNinja will help create a single source of truth for end-to-end testing and improve the quality of tests and the end experience for the user.
Interconnected applications increase the complexity, time, effort, and cost of end-to-end testing. Third-party services connected to your software further increase the cost of running tests, especially when you have no control over those services and they also have a chance of failure.
After reviewing some of the most common pitfalls, we look at a few methodologies to enhance end-to-end testing.
Due to the abovementioned issues, you must keep end-to-end testing at the correct balance with the unit and integration testing. End-to-end testing does things that other forms of testing don’t do, ensuring that common user workflows perform as expected. While it is key to testing common user scenarios and ensuring the application’s stability, other types of testing, such as user testing, are more suitable for isolated bugs. Therefore, relying solely on E2E is not enough.
Google recommends 10% end-to-end tests, 20% integration testing, and 70% unit tests. This 10/20/70 split is a good baseline for you to work from, but naturally, it will differ based on your product and team. Keep in mind that an inverted pyramid suffers from many of the pitfalls we’ve outlined, and an hourglass shape probably means you’re using end-to-end testing when you should be using integration testing.
Sometimes, the issue wasn’t a fault in the software but rather a system failure that can happen regardless of changes in your code. OS, browser, or hardware crashes may cause your tests to fail, even though it has nothing to do with your code. When performing automated UI testing, it can be helpful to record the testing sessions, either with video or a log of operations.
Recording tests in such a way helps identify the cause of the issue.
Automation tools can help reduce issues such as misconfigurations, which introduce much of the flakiness in end-to-end testing. Equally, automating key tests as part of your CI/CD pipeline will make it easier to ensure security across all stages of the development process, without the extra burden on developers.
Flaky tests are a severe hindrance in end-to-end testing, but you can mitigate the effect of flaky tests by disabling them. Identifying which tests are flaky requires that you perform a large volume of tests, which is an issue in a relatively slow process as end-to-end testing. But if you identify a flaky test, disabling it until you can fix the test is often the best course of action.
End-to-end testing is not the be-all and end-all that it seems to be. There are many pitfalls to avoid, but also just as many solutions. Make sure you have a good balance between end-to-end testing and other more reliable, quicker forms of testing, and you can get the most out of it. If you’re interested in how Tabnine can help you make testing more efficient, book a demo to see how it works.
Are you tired of the endless sprawl of documents across your organization? Do you need a tool for document management, knowledge sharing, and collaboration? Confluence may very well be the tool you’re seeking.
Confluence has been around since 2004 and is used worldwide by over 60,000 organizations. There are numerous use cases for Confluence, and it seemingly fits in with every department in your organization. Let’s clarify the confusion around the different use cases of Confluence and find out if it is the right tool for you.
Confluence is a wiki-like collaboration and information management tool developed by Atlassian. We say wiki-like because everyone is familiar with Wikipedia, a website based on MediaWiki, so it is easy to grasp what it could do.
Big teams gather and produce an endless array of information. In teams that don’t use a platform like Confluence, this knowledge base must be passed by word of mouth, which is inefficient and produces inaccuracies.
Unlike MediaWiki, Confluence is designed for enterprise collaboration and management, with some key features that large teams will find invaluable:
You can host Confluence in any of the typical choices afforded by software vendors targeting enterprise organizations. The easiest solution is SaaS, where Atlassian hosts and manages the software for you. Atlassian has a free cloud plan, which means that even if you intend to host it yourself, you can get to know the product before purchasing it.
You can host Confluence on IaaS or locally for a more involved solution. Regarding IaaS, you can easily host Confluence on AWS or Azure and retain significant control over configuration to adhere to your specific needs. Local hosting of Confluence requires higher maintenance but could be the right choice if you require tighter access management and data governance.
Confluence is a product that solves a wide array of problems across many potential fields. Any enterprise organization is bound to accumulate a wealth of knowledge that must be passed around between different teams and used to train new employees.
Engineering teams are most likely to use a technology-oriented tool like Confluence. Still, other departments, such as Marketing and Product Management, could benefit from having a shared knowledge pool, especially when you can cross-pollinate with other departments.
As a tool, Confluence has a set of features that trivialize specific use cases. Atlassian offers Spaces built from templates for several common uses, which makes for a fast setup:
Technical documentation – Writing technical documentation as a separate set of documents causes repetition, errors, and disjointed information. Confluence has features that prevent unnecessary repetition by making sentences, images, and other segments reusable.
Knowledge base – IT teams need quick and easy access to information when helping customers solve problems. Having a fully searchable knowledge base and quickly adding information to the knowledge base will allow your IT department to focus on the client.
Intranet – A central hub for your organization where users can be added without administration overhead helps everyone stay connected. Setting up a space for each team is a sound basis for an intranet.
Software development – The most evolved aspect of Confluence is the tools available for software development. With many templates to choose from, a software development team can get a head start on vicarious common topics such as:
While it’s true that Atlassian put a lot of thought into Confluence and the above use cases, it doesn’t mean that they are perfect solutions. How can you tell if Confluence is the solution for you?
There are many suitable applications for Confluence; here are some common ones.
Every organization, from the smallest to the largest, produces documents. When the number of records is small, it can be manageable to spread out across multiple Google Docs or Word files in a Dropbox. But as documents accumulate, they become increasingly challenging to maintain.
Confluence has various tools to help you manage documents, which is the process of storing, archiving, and retrieving documents. You can upload documents to Confluence and attach them to pages or create documents inside Confluence. You can have document trees, which help associate documents with concepts, departments, or teams. Confluence can also link documents to various stages in your workflow if you’re using Jira for product management.
Collaboration is a core feature of Confluence. Commenting and mentioning colleagues can all be done in real-time with multiple contributors. Feedback loops are essential to creating, maintaining, and improving documents. Collaboration features are critical when various teams or departments need to process, review and maintain these documents. They also enable higher productivity.
The Atlassian website has countless templates that can fit every need. There are templates for design, Business Strategy, Customer feedback interviews, Product Management, Human Resources, and many more. A short browsing of the templates will spark ideas for how to use Confluence across all your departments.
Atlassian may have adopted Confluence for nearly every document requirement, but that doesn’t mean it is the best tool for the job or even an adequate one. Here are some tasks for which Confluence may fall short.
As a developer, you are constantly searching the web for solutions to problems, which is why it is unlikely that you will choose to search Confluence for an answer unless you specifically know it’s there. Unless you’re the person that wrote the code documentation, it is unlikely that you will find it in Confluence.
Code documentation should exist inside or beside code, which means code repositories and comments. Effective documentation is part of your workflow, not outside your workflow. Just like Tabnine’s code completion helps developers with AI-powered code completion right inside their IDEs.
Your code writing becomes far more efficient when you don’t have to leave your IDE to search for syntax, code snippets, or simple solutions. You wouldn’t want to leave your IDE for code documentation, would you?
There is no doubt that Confluence is a feature-rich solution, but when features hit a critical mass, they become feature bloat. Editing documents in Confluence is prone to mistakes, sluggishness, and an overall challenging experience.
A user writes: “An issue that users can face when using Confluence is attempting to edit a document while someone else is editing. Although users can access the document and save it, they cannot see the changes happening in [real-time] that other users are implementing until they refresh their page. Some users have also noted that this can result in loss of edits.”
Knowledge management and document management aren’t synonymous. Managing knowledge is about linking information, making it easy to find when needed. The information you seek might be spread over several documents or be a small section of a document on a different subject.
When you need a specific document, it is easy to find it on Confluence, but the search functionality will leave much to be desired when you’re seeking knowledge. And because you may archive knowledge for long periods, when you do manage to find it, you may learn it is outdated information.
If you’re using Jira, you’ll likely want to jump in on Confluence as the pair work well together for product and document management. While the template page may give you the impression that Confluence can do everything, things like knowledge management and code documentation are better left for other tools. Check out Tabnine AI-powered code assistant for better team collaboration and more efficient code writing.
Tabnine Enterprise is an AI code generation tool that helps software engineering teams write high-quality code faster and more efficiently, accelerating the entire SDLC. Designed for use in enterprise software development environments, Tabnine Enterprise offers a range of features and benefits, including the highest security and compliance standards and features, as well as support for a variety of programming languages and IDEs.
Both Tabnine and Copilot offer AI coding assistant solutions for R&D teams. So which solution is better?
We’ve put together a comparison using the most common parameters that we’ve encountered in our work as developers, while also serving the dev community for several years. From our POV, these parameters reflect the most important needs, pain points, and challenges that must be addressed by AI code assistants for enterprises:
[table id=5 /]
Let’s look at the parameters and how each solution compares.
Tabnine Enterprise offers full privacy for its customers’ code:
This ensures you always have complete control over your IP and protection against IP leakage.
On the other hand, Copilot for Individuals and Copilot for Business send both code snippets and user engagement data to Microsoft.
With Copilot, it’s possible to opt out of saving code snippets; however, user engagement data is collected and used to improve Copilot’s AI models.
The code on which a solution’s AI models are trained can have a serious impact on the companies that use the solutions.
Tabnine doesn’t train (and never has trained) its AI models on code with nonpermissive licenses and offers full transparency and attribution. This means that Tabnine isn’t restricted by the GPL license’s copyleft provisions, and protects users and customers from possible related consequences. In addition, it’s always been Tabnine’s goal to honor the intent of code authors and maintain good faith with the rest of the developer community.
On the other hand, Copilot trains its models on OpenAI, which could result in legal exposure for its customers. This is because OpenAI is trained on nonpermissive open source, such as GPL, and doesn’t disclose its training set or provide references. Copilot does offer a filter to remove code suggestions that contain problematic code; however, it’s not always effective, meaning that the legal risk remains.
While Copilot is trained only on OpenAI, Tabnine Enterprise allows customers to connect Tabnine’s AI models to their code repositories. This means the models, which are only accessible to the customer, can learn the organization’s coding best practices, styles, naming conventions, and more, resulting in code suggestions that are more context-sensitive and relevant. In addition, this allows for faster onboarding and training of new team members and junior developers.
Both Tabnine Enterprise and Copilot Business offer central management and configuration; however, the features offered are different.
Tabnine Enterprise’s centralized configuration allows customers to:
Copilot Business’s centralized configuration allows customers to:
Copilot for Individuals doesn’t offer centralized configuration and allows customers to:
On the surface, the price appears to be almost the same ($20 per Tabnine Enterprise user, $19 per Copilot Business user).
However, Copilot Business is only available to companies using GitHub Enterprise, which charges $210 per seat per year. In effect, that’s a barrier for enterprise companies that don’t need or want GitHub Enterprise.
Copilot for Individuals costs $10 per month (or $100 per year). However, the Individual plan doesn’t include any of the features suited for enterprise, including privacy and security.
Both Tabnine Enterprise and Copilot Business allow enterprise customers to configure and manage user roles and permissions. However, Copilot for Individuals doesn’t offer any user management capabilities.
Whether you’re choosing an AI coding assistant to make your life easier as an individual developer or choosing a tool to deploy to your entire engineering team to improve the effectiveness and satisfaction of your team, it’s critical to evaluate potential vendors holistically:
Only Tabnine meets all of these requirements expected by enterprise engineering teams and has the history and scale of developer adoption to prove it.
Since launching our first AI coding assistant in 2018, Tabnine has pioneered generative AI for software development. Tabnine helps development teams of every size use AI to accelerate and simplify the software development process without sacrificing privacy and security. Tabnine boosts engineering velocity, code quality, and developer happiness by automating the coding workflow through AI tools customized to your team. With more than one million monthly users, Tabnine typically automates 30–50% of code creation for each developer and has generated more than 1% of the world’s code.
Unlike generic coding assistants, Tabnine is the AI that you control:
Tabnine ensures the privacy of your code and your engineering team’s activities. Tabnine lives where and how you want it to — deployed as protected SaaS for convenience, on-premises for you to lock down the environment, or on a virtual private cloud for the balance of the two. Tabnine guarantees zero data retention, and we never use your code, data, or behaviors to feed our general models.
Tabnine is personalized to your team for optimal impact on your business. Tabnine is built on best-of-breed large language models (with the flexibility to switch as new models emerge or improve) and gives you the ability to fine-tune or deploy fully customized models. Tabnine is context-aware of your code and patterns, delivering recommendations based on your internal standards and engineering practices.
Tabnine works the way you want, with the tools you use. Tabnine supports a wide scope of IDEs and languages, improving and adding more all the time. Tabnine also provides engineering managers with visibility into how AI is used in their software development process and the impacts it is having on team performance.
Tabnine is secure and trusted. Tabnine believes in building trust through algorithmic transparency and thus shares how our models are built and trained with our customers. Furthermore, we’re relentlessly focused on protecting our customers’ interests by only training on code with permissive licenses and only returning code recommendations that will not be subject to future questions of ownership and potential litigation. We respect open source code authors and their rights as well as the rights of every one of our customers.
As you should expect from any vendor, Tabnine offers proven, enterprise-grade security and meets key industry standards.
Get started with Tabnine for free for 90 days, or contact us to learn how we can help your engineering team be happier and more productive.
