Cursor for Quality Assurance

0:00 Thanks for joining the session today. We're super excited to kick this off. Before Ryan goes into the demo, um I'm just going to give a quick background on Cursor and an overview of the platform so you have a little bit more context about what you're going to see. Um so Curser launched in 2023, but in this really short period of time, it's become the standard for AI powered software development. And the reason for that is that we're bringing together developers,

0:29 tools, and AI agents into this single system that plans, builds, and ships code faster for your team. And really, our mission here is to build the engineer of the future. And that's where human developers and AI agents can work side by side to really supercharge productivity across your entire organization. For those of you that um have been [clears throat] familiar with Cursor already um you know that it started as an IDE built on VS Code. Um but really in the last year it's it's evolved to be

0:59 so much more than just an editor. Firstly, Cursor has full semantic awareness of your entire codebase. That includes things like files, dependencies, and team conventions. And on top of the editor, we also offer native AI workflows. things like plan mode, which I'm sure Brian will cover today, um, so that teams can structure really structured project plans, um, and start building as well, all in one place. Beyond just writing code, Curso also addresses really common bottlenecks, um, in the SDLC. So, our

1:34 bugbot is our automated code review agents. Um, and that helps teams catch issues as early as possible. Um, to help with making improving quality of code that's going into production. We also recognize that not all developers um are working inside the IDE. So, cursor is really designed to meet teams where they already work. We offer a CLI for folks that prefer to work in our terminal. Um, and then we also have cloud agents that can accomplish async tasks in the background while you're working in either the

2:08 terminal or the IDE. In addition to this, we also have integrations with tools like Slack and Jira and your CI/CD stack. So AI can really support um the entire software development life cycle. What ties everything together in cursor is context. I had mentioned at the top that um you know every part of cursor operates with a deep up-to-date understanding of your entire codebase and that's really really critical for uh larger complex projects.

2:38 And in addition to this, cursor is model agnostic. So we give our customers access to all of the frontier models directly in our platform pretty much as soon as they're released. And we've also trained our own um models inhouse that are purpose-built for software development. So they're really really smart um and also really fast at building.

3:00 And all of this together is really what differentiates cursor as a platform that's going to embed AI across the entire SDLC. And as you're all familiar with um you know teams for decades have really worked in this linear cycle where it's plan design code test deploy monitor and cursor touches every single stage of this. It connects to Jira linear and notion to generate specs. It pulls from Figma to create implementation ready designs and it uses AI agents to explore the best solutions during development.

3:32 Testing and QA like I mentioned before they're all automated through Bugbot while deployment integrates directly with CI/CD and observability tools like data dog New Relic and Splunk. And the result here is a continuous loop where intent flows from planning into production.

3:51 All right. So, just wanted to share that big that quick background on cursor and I'm going to pass it off to Ryan who will jump into the demo now. >> Great. Uh, thanks for that Lauren. Uh, great background. Um, so yeah, uh, my name is Ryan. I'm on the field engineering team here and uh today I am going to go through a couple of workflows for folks that are working in QA or software engineers in test to kind of optimize their day-to-day in terms of

4:18 writing tests uh understanding test coverage and uh just kind of building systems to help you uh you know uh work through uh fixing bugs and understanding bugs and and working in uh larger testing suites. And so um throughout this whole demo I'm going to go through different use cases. Please feel free to add as many questions as possible. I'll try to pause at reasonable times so we can pick up some of those and then Lauren, if there's anything that comes

4:40 up, uh, feel free to just interrupt me and we can, um, you know, address them as things come along. >> You got it. >> All righty. So, me share my screen. Okay. So, uh, a couple of things uh, before we get started. I'm just going to go over the, uh, project that we have open in cursor today. Uh, we'll be going over two different projects. So, one is a little bit more fun uh and silly and and and still has some very real enterprise use cases uh for unit testing. And the other

5:08 one is a larger legacy codebase uh in which we're going to um look at kind of strategies on how to do some sort of refactoring and creating guardrail tests for much larger projects. And so, uh it's important to keep in mind that uh cursor has a very deep understanding of your codebase with our uh underlying semantic indexing strategy. So it doesn't matter if it's a smaller codebase uh that is newer or if it's a larger codebase that maybe has been

5:31 around for 10 years with a lot of files, you know, potentially gigabytes of files. Cursor works really well within code bases to understand the underlying architecture and to pull in relevant information using semantic search and uh efficient GPS and everything that comes into our agent hardness. So um that's kind of one of our key differentiators from a lot of the other tools on the market. uh we're able to do this really effectively in both our synchronous uh

5:54 uh features uh like our IDE and also our asynchronous features like our bugbot and cloud agents products. Okay, so uh let's get started here. The first thing I want to cover is um our uh how to think about doing unit testing within cursor. So here is my application. I have this little uh full stack application. It's a Python fast API backend VUJS front end and really it's an app I built to find adoptable cats in Brooklyn where I live. So I can filter

6:25 by uh different different filters here for different uh features of these different cats. And I have a little statistics page. So we're going to be building off of this and I already have a couple of unit tests set up for this application. And what we're going to do is we're going to add some more and we're going to think about how to add unit tests in a way that feels consistent. So for folks that have used cursor before probably one of the first

6:46 things uh any developer wants to do and maybe uh wants to optimize in their workflow is writing unit tests. I think as developers no matter when we started in our careers probably was our least favorite thing to do. I don't think I have ever met a developer who looks forward to writing unit tests. It's usually not something we want to do. And so um having AI agents help us write code uh allows us to kind of get rid of some of these more menial or uh uh

7:09 boilerplate tasks that are not super exciting to us. And so uh you know one approach is just to ask the agent itself in agent mode to write unit tests for us. However uh if we do that we're relying on the agent to kind of pull in uh relevant information from the codebase understand the patterns and then use those patterns to write repeatable tests in the way that they've already been written. And I actually have a couple tests here in my uh backend tests folder. Let's go to this test main folder. So I have some tests

7:38 with some existing structure. And actually this structure is pretty repeatable and the agent will probably do a good job at just looking at these tests and understanding them well and then uh also uh implementing them in the way that they've already been uh you know shown to be implemented. And so this is one of the benefits of cursor. You don't have to give it that much guidance. The agent can kind of figure it out based on pulling in the right

7:59 pieces of context or information from the existing tests. However, there are going to be a lot of use cases where you are writing unit test that maybe there are some that require some sort of level of extra convention either from your team's unit testing practices or even a set of configuration steps that need to be run outside of the the test code themselves. Like maybe you have to run uh the um testing command with some parameters uh based on some JSON config

8:25 or you have to like deploy them remotely to test out some other uh um development servers that need to be spun up. there's a lot of exotic ways to uh write unit tests and every organization has different ways of doing it. So one solution here to kind of keep the writing of tests more consistent and to obey certain conventions is to create a cursor rule for it. So if I go into my ccursor directory at the top here you can see that I have this rules folder

8:49 and so this rules folder has a set of different uh MD files. These are effectively just markdown files, but MDC is cursor's way of distinguishing a normal markdown file and making it a rule file. And really all it does is add this front matter to the markdown right here. Um, but here I have a backend unit testing.mmdc. I have my rule set to apply to specific files. That's a setting that I can um that I can select in this underlying dropdown. And I have

9:16 it set to look at the backend test directory. So, what this means is I'm telling the agent every time you are going into our back-end test directory or you're operating around that directory, be sure to pull in these rules or this file into context and use this to guide all of your development.

9:34 And so here I have a lot of rich data on how my unit testing framework is set up, right? I'm using a certain piest framework. Uh I have a test structure that I want to be adhered to. So if the agent decides to make new test files completely, it's not going to choose something completely different. it's going to adhere to the structure that I've that I've set it out. I have naming conventions and then I even have sample examples of how to test different types

9:55 of of endpoints here and different types of functionality that I have in my codebase. And so now when I write unit test, it'll be consistent with the um the structure of the rules that I've set out. So uh we'll go ahead and test this right now. Uh so I'm going to say add some new unit tests to my back end. So very vaguely I'll say this. Um but what we'll see is that the agent and actually I'm going to use our composer one model because it's quite fast here. Uh I'm going to

10:30 use the the what the agent's going to do is it's going to uh you know navigate the codebase and you'll see that the first thing it does is it reads this backend unit testing.mmdc right here. So, it's pulled in our rules for our given task, and now it's going through the existing file, seeing what it needs to add, and now it's updating some tests in our main.py, and we should be able to see a few more tests uh that are ideally increasing our test coverage here. So,

10:54 here we have a whole test on the pageionation in my app and then also a test on some edge cases, which is kind of cool. Then, it's adding some new ones in our test order. And so all of this is adhering to the the same framework that I uh or the same rules that I added in my rules file uh the conventions for my unit testing. And so uh the benefit here is that now when new developers come into my app, they want to load it up, they want to maybe make changes to it,

11:19 add new tests, this rule will allow their agents to be consistent with the existing principles and conventions in the codebase already. And so it's a really nice way to kind of add a resource within the codebase that kind of uplevels the health of the codebase as we move forward with new development.

11:34 So across different developers uh and across different life cycles as well. Uh another thing to note about rules is that what I created here is what's called a project rule. So a project rule exists within the project in the codebase hence the name. Um and so this is really specific to um your existing project that you're working on. If let's say for example your organization has multiple backend uh code bases with the same kind of you know language like

12:00 maybe they're all Python backend code bases and you want to adhere to the same principles of unit testing across all of them what you can do is you can create a team rule. So a team rule allows you to add rules on the organization level that can be shared across all of your teammates. So if there is ever something that you kind of need to abstract out to to share across the organization uh in terms of testing practices, you can do that with a team rule. And we're going

12:22 to go through skills in just a little bit um in the context of te testing. We'll actually have a way to share those as well uh across the team. So uh we can see here that um we created a bunch of new tests and then also in my unit testing rule I noted how to uh uh I I told the agent how to run my test and so it ran the test using the piest uh command and it verified that everything passed accordingly. So now we have a whole bunch of tests. So we're going to

12:48 keep these. There we have it. So, uh, pretty simple. I mean, the agent is extremely powerful out the box, but you can augment that kind of standardization, uh, with with cursor rules here. Um, so I'm going to go ahead and pause just, uh, just before I before I get into our next topic and see if, uh, there are any questions that we could answer that are related directly to this. >> Yeah, it looks like there's a few questions coming in on the chat. Someone asked about uh, what about test-driven

13:17 design? Should we be writing the test first going from red to green only adjust test when the TDD was naive? >> Yeah. Uh this is a great uh a great like uh comment with a bit of foresight. So uh one of the examples I'm going to go through is kind of a a way to approach TDD in the age of agents. So really uh when you're working with agents, they really work well when you specify requirements up front. And so TDD as a concept exists way before uh agent

13:47 coding but it's really well suited for this use case. So if you can specify what the functional requirements are for uh you know creating a skeleton of tests before implementing the logic that can be a really good guiding light for the agent to implement the logic uh afterwards. So starting with tests and then going to implementation is a really good uh methodology. And so we're going to cover kind of a version of that in a little bit uh using our plan mode. And

14:10 so plan mode is a good way to kind of structure out that process. Uh and uh yeah, so I think I think you're whoever asked that question your mind is in the right place. I think that's a very effective strategy. >> Yeah. And other folks are curious about rules. Um so how you created the rules?

14:25 Is it done through the agent? Can you define rules? And how do you set the rules? >> Yeah. So I actually uh cheated. I didn't write this rule myself. I used slashcreate rule. So uh these are examples of default skills that uh you're able to access within cursor. So if I do slcreate rule, let's say for example I wanted to create a generic API development rule. I could do slcreate rule and say um uh create a generic API rule based on at uh main.py which is my where my API is

14:58 and the agent will be able to generate rules for you uh based on that. So here I generated a rule based on my existing unit test and I also added some extra logic in there uh by asking the agent to update the rule as well. So really useful for taking existing logic and encapsulating it in in a rule.

15:14 >> Yeah. And it looks like folks are um also looking for you to show a way to write end to end UI tests as well. >> Right. Everybody's everybody's uh reading ahead, reading my mind. Okay, perfect. >> All right. [clears throat] So the next thing I'm going to cover here uh is this concept of skills. uh and then we're going to move into a more endto-end example. So uh we've talked about cursor rules. So cursor rules are ways of like guiding the agent with a kind of standard or I think I like to think of

15:39 it as a manual for doing certain operations. In this case it's writing unit test. Uh it can be anything. It can be you know as long as you uh configure the rule to be set in such a way that it either relates to files in the codebase or you can have it always apply imply intelligently or you can trigger them whenever you want. Um but these things you can kind of think of them as like static resources. So the agent is going to pull them in when it's in a certain

16:02 area of the codebase or when it um you know uh or it is get gets pulled in all the time, right? So it's only pulled in on kind of like a um on an instruction basis, right? So when the agent wants an instruction to do a task, then the rule is used for that. If you want the agent to do a certain action, however, maybe it's something a bit more active tense. If it's like a workflow, uh that's a good use case for a cursor skill. So a cursor skill you can think of it as like

16:29 a tool in a toolbox. The agent is a construction worker. The rules are the manuals or the uh you know the city code that the construction worker has to obey. And the tool is a set of uh or the skill rather is a set of tools in a toolbox that the agent can select from when it sees that something is relevant to the task at hand. And so actually I have a set of skills here uh that are that are kind of interesting. So uh if you go to the docursor directory add a

16:54 skills folder you can add a skill by creating a subfolder uh called with the name of the skill that you want to create and then in that subfolder you create a skill.md and also in that subfolder you can create a scripts subfolder. So what what is going on here? So a skill is a combination of a markdown description in natural language to the agent and an explicit uh deterministic executable script. You don't always need the script. You can have a skill just be a markdown. But the idea here is that there are certain

17:22 processes that you want to do over and over again that have a um natural language piece where you want the agent to kind of think about something, process some information using its intelligence. But then there's also a part of it that you just want the agent to do something, right? Something deterministic, something that you can rely on each time. You don't want it to come up with a new piece of code or a new solution. You just want to run a script. So in our case here, I have this test coverage skill and I can trigger it

17:46 in two ways. I could just hit slash test coverage and uh pass a parameter to it or I could just tell the agent, hey, can you check my backend test coverage? Either situation. In the first situation, I'm explicitly triggering it triggering it. In the second situation, I'm having the agent dynamically look for the skill uh based on the the user's prompt and then pull it in. And so that's the benefit of skills. You don't actually need to directly reference

18:11 them. If the agent is aware of the skill, it will pull it in and use it where it feels it's convenient or useful for the task at hand. But here, I'm just going to directly trigger it uh for our example. And I'm just going to say test coverage backend. And if we read the skill here, you can see that I've given it some instructions on how to do coverage reports for both the backend and the front end. And you can see here part of those uh uh those uh those

18:35 examples are are to uh run the underlying coverage.sh SH which runs like a a piest uh coverage command. And then the other part is to just um you know how to output and and uh format the the existing report as well. And so here it ran our output uh or our coverage is 100% since we added the uh uh added our new rules which is nice. We got a a report of all the tests passing and we also have an HTML report. So actually let's see if we can take a look at that

19:03 here. Um open in browser. I'll open our integrated browser and we should be able to see it. Okay. I am not a robot, Google. All right. Um, let's try this guy. Okay. Well, I'll uh I'll go ahead and uh we can we can pretend that it it it displayed, but it did create a report and it came up with the with the data here. Uh if we have a bit time, I can come back and and view this. So, um, the the main, I think, thing to show here is that you can use skills as kind of package workflows. So, things like

19:43 testing coverage and maybe even doing something off of the coverage. Let's say you want to push this coverage report up to Confluence or something. We can use MCPS to kind of go back and forth. Uh, in fact, uh, I actually I think I already have our Atlassian MCP connected here. So, if I go to my settings, uh, one thing I can do is I can just push this up. So, I'm just going to double check if I have my MCP. Refresh this guy. And then I'll say uh upload the

20:08 coverage report to Confluence in the cat stats space. So, that's the space for my application. And so the nice thing here is let's say as part of the uh QA process you want to upload the coverage report, share it with your colleagues and then use that to make tickets off of it and potentially uh distribute a load of work. You can take this coverage report uh process it and then uh uh push it up as a report as well. And so here uh the agent is going to um you know uh look at the the page and then add a coverage

20:44 report here uh to update our confluence page for our project. So we'll come back when this uh when it actually makes the change here. Okay. So uh the next thing I want to cover is a little note on or or kind of uh shift gears from unit testing and go to integration testing. So actually before doing that looks like we have our confluence page. So let's go over here.

21:12 So if we scroll down here we go. This is our latest coverage report. zero failures, execution time, uh the code coverage summary of all of our different files in the back end, and then we even have like uh extra test results down down below for each of the different uh tests. So, pretty nice. I mean, uh now we have a resource that we can share with the rest of the team to use to do other work. Okay, so integration tests.

21:41 How do we think of doing integration tests uh in cursor? Well, integration tests ultimately are just files, right? I mean, even though they're doing more complex operations than unit tests, they're ultimately just uh interacting with some sort of external process. Uh whether it be a web server in uh you know, in a browser like this or whether it be a deployed uh platform or an executable, um integration tests are testing uh you know, interactions

22:08 between one system and another and are testing things that are a little bit more complex to represent in simple uh unit tests. And so uh the example I'm going to give here is twofold. One, I'm going to show how the agent can kind of prototype a type of action uh in the browser. Uh so being able to do a set of clicks uh do a set of actions and uh kind of get the metadata from that and then use and then the second piece is I'm going to take that set of actions

22:33 that the agent does and use it to create an integration test for us. And so uh the nice thing here is that our integrated browser actually has playright under the hood. So we can actually use the agent to do complex things like click buttons or navigate to pages even take screenshots and then based on that data we can create a robust and rigid integration test that we can then deploy and use for you know testing this in staging automatically for example. So the action that we're going to do here is what I want to do is

23:04 I want to uh have the agent navigate and test the flow of browsing different cats. So, I want to click I want the agent to be able to click browse all cats to filter down by a certain age. Maybe we'll say senior cats. Uh and then I want the agent to scroll all the way down until uh 30 items are populated as an example. So, relatively complex task. I mean, maybe not that complex, but uh something that I think the agent can do in multiple steps. So, I'm just going to

23:33 phrase that uh to our to our agent here. I'm going to say um navigate to say using the browser navigate to the homepage. Click on browse cats. Cats uh filter by senior cats. and scroll until 30 cats are populated. Take screenshots to show your work. Okay. So now what the agent's going to be able to do uh similar to that same MCP pattern we mentioned before is uh the agent's going to be able to understand hey I need to access the browser. So, I'm going to use our integrated browser tool to do so. And then I'm going to do

24:38 a set of actions in the browser and then uh report them back to the user. So, you can see here we have this little banner on the right hand side saying agent is using this browser. It's doing all of the actions we asked it to do, right? It locked the browser, click the browser navigation link. As we can see on the left hand side, there's a a bit of a uh um there's a log there showing that. Now it is waiting for something to load, capturing a screenshot

25:04 and then now it's doing each thing step by step. So the benefit here is you can actually see see this workflow in action. Uh at each of the steps um we have the metadata of the actual action being captured by the underlying playright integration. You can also in the prompt uh have it output some logs to some text file if you want to really record each of the elements uh on the page that are being accessed. uh but you know whatever use case you have you have

25:29 the ability to kind of extract information from this process and then use uh use this information to actually uh create an integration test. Um and because we learned about skills this is something that we can do over and over again. Say for example you want to keep doing processes like this. We can create a skill called uh browser automation. And that browser automation skill can take in a user prompt and then do all of these actions and then maybe even create

25:54 an integration test off of that. So actually I'm going to do a follow here. I'm going to say based on on this action create an integration test test in our suite. So this is cued. We're going to wait till this this finishes. I think it's going to take a little bit of time to scroll down. And now it's checking the snapshot. How many cats are loaded?

26:29 Okay, it loaded 60 cats. So it should be done. So just to give a bit of a preview. So we have this front-end repository or rather front end directory on the right hand side. I already have a set of integration tests here. So I have like this filter integration test. So we should see another integration test being added as the followup.

26:54 Okay. So I think it it will be done. It's giving me the summary. Great. Loaded around 60 cats. Perfect. And then now it is doing the second piece. And so um it'll look at the history of all of the actions done. We have some metadata on the different things that have been clicked and so it should be pretty easy to uh to create our integration test here.

27:22 Cool. Uh while we're waiting for this to to finish, any any questions from from the group on ede testing and and how uh how the agent can help with this this process? I think there were a couple questions from the group >> [clears throat] >> um that went a little bit further back.

27:44 I've been u monitoring the Q&A as well. Can we do accessibility testing by cursor? >> Yeah. Yeah. So uh you definitely can. I mean accessibility testing usually uh I guess there are different ways it's done but if you have a set of the uh standard accessibility requirements uh just having that even as a markdown file and giving it to the agent saying like hey can you run through these requirements and verify that all of these uh um features are present in the application cursor actually does a really good job

28:12 and uh I worked with a major uh um one of our major clients who's a large insurance company that used this exact use case to um go through one of their their UI products. >> Awesome. Um, and then someone said, "Never trust an automated test that you didn't see fail. You write tests based on existing code. This is dangerous how you address that. Uh, you have no proof.

28:35 The tests actually the test assets actually um I'm sorry, this actually show something." >> Yeah, I think I get the gist. So I I think uh what the user saying is like you should never trust tests that are created and just automatically pass. Yeah. I mean I think yes the answer is yes you shouldn't. Um I think for the purpose of the demo it's good to show like tests being created you know quite easily. However it's always good to validate what it is you're testing

29:03 right. So I think the important thing is you know cursor here is an accelerant to your existing engineering process. So if part of your engineering process is to make sure that hey I just outputed this functionality, I need to figure out what exactly a proper like golden state or a proper happy path of this functionality is determining that explicitly and then making sure that the tests that you're writing even through the agent are going to verify that that finishes. And so

29:30 whether that's coming up with the happy path data yourself and then having the agent write the test with those kind of uh those resources as the uh input and output state or whether it is coming up with a detailed plan and having the agent adhere specifically to that plan and then verifying it by even just manually checking the output code. Either way works. uh it's just important to practice good engineering principles and and look at the underlying code that

29:53 is being written and having a way to validate that uh new code that you're adding even if they're unit tests are matching the requirements that you you set out to put in your in your project. >> Awesome. Um and then one more thing is please share how to automate endtoend UI tests where we need to have access to a third party data um like with mule as the integration layer. Yeah, I I don't have a demo to um to show that exact integration path, but essentially um as

30:22 long as the agent can access the underlying code and the agent can access the um the files or the instructions to spin up or spin down whatever other services need to interact. It'll be able to understand, you know, like let's say it's an API integration that communicates with a third party service.

30:38 there's probably a shared layer where uh there's a contract between you know what schema is being used to kind of send the a uh receive APIs or like send a request to the external server and then consume it. Uh you can pretty much create a script to send those API requests yourself or have the agent send them and then based on the response schemas create a a test to validate whether those are correct or not. So uh one example is just you know having a a

31:05 Python script that uh fires off an API request something comes back you have like maybe pyantic or some sort of model validator to make sure that the request is as expected if it's not as expected then the test fails. Um cursor can do that in the same you know level of ease it did uh the test that we're showing uh generated right here. So uh you should just use the same kind of uh strategy as we're showing to do that piece and it should work.

31:31 Okay. Um I'll let's keep chugging along here. So the next topic I want to cover briefly is how to think about uh a bit of root cause analysis uh in cursor. So let's say for example you have a bug that's being reported and you want to do some ground work to identify what that bug is. Maybe you don't necessarily want to make the bug fix, but you want to kind of you know get closer to the root cause so you can pass it over to uh the engineering team in charge of that

31:57 product or maybe take a first stab at fixing the bug yourself. So uh to that effect, we have an agent mode called debug mode. So if you go into our agent selector, there's a couple different modes here. One of them is debug. If I click into debug mode, this kind of shifts the focus from the agent mode uh where you know from the agent mode that just kind of outputs code based on input context. So the way the agent works is it does search and retrieval. It finds

32:20 the relevant context from the codebase, thinks about the problem, and then immediately works on doing a set of actions to solve the problem. whether it's running shell commands or writing code, it kind of has a bias for action. And in general, when you're doing bug fixes or you're giving the agent like a stack trace or you know a bug report, the agent will try to find the part of the codebase where it believes the bug is originating from and make a code

32:42 change, right? Its bias is to make code changes. Now, for most bugs, that will probably solve them. Agents are extremely intelligent because the underlying models are extremely intelligent. But for very subtle bugs, especially bugs that maybe exist across multiple services or bugs that are kind of um you know kind of nested between layers of logic, a lot of times you need a more um fine grained mechanism to find the underlying root cause. And so debug

33:06 mode is a way to get to that. So the way debug mo mode works is it's quite simple. uh it essentially just adds log lines in parts of the codebase to uh to determine how the action path of the uh the user's actions is um you know is going through the codebase and looking at that action path and then determining what the root cause is from the underlying logs. And so what debug mode does is it creates a lightweight web server initially. uh that web server uh

33:33 its sole purpose is to accept requests and then debug mode comes up with four to five different hypotheses for the underlying problem based on the user's description. Based on those hypotheses, it'll instrument the codebase with log lines. It doesn't matter where in the codebase it is. It could be uh you know across multiple services. It could be you know um backend or front-end code.

33:54 It doesn't matter in doesn't matter the language either. It just needs to have the ability to send a uh like a JSON to the underlying server and then based on those log lines, it'll instrument them out. After it creates those log lines, it'll ask the user to reproduce the step. So, reproduce the bug, right? Uh and then when the user does the reproduction steps, now the agent has access to a set of logs that are detailing every single part of that

34:18 execution path in the codebase. And then based on those logs, it'll look at the hypothesis, decide which hypotheses are valid or invalid. And then from there, it'll make a very surgical targeted uh bug fix. And then that way you're not relying on the agent to kind of make a big guess and maybe change some parts of the codebase you didn't want it to change. You're relying on the agent to just look at a very specific part of the codebase and only fix what's broken,

34:42 right? And then after that uh the agent has made the bug fix, it will ask you to verify that the bug has been fixed. If you verify it's been fixed, then it will just remove all the log lines. So, it's a pretty simple approach, but it is applicable to uh a wide variety of uh types of software that you can build, not just full stack applications, but even applications that uh are compiled into executables. Uh as long as the application can fire off logs to some

35:07 server that's uh collecting them, then this approach will work. And to demonstrate this, we're going to do a pretty simple bug, but uh one that is at least a bit visual. So for here is uh so for my cat data right I have a bunch of these cat listings that don't have images the underlying image provider maybe the the the cat listing is outdated or the image is lost that's totally fine but what is unexpected is that uh we're actually supposed to display this icon in the uh for some of the cat images but many of them don't

35:36 have this icon right so this is definitely a bug there's some inconsistency here so we're going to use debug mode to try to figure out what the issue is I'm just going to say Um, some of our cat images that aren't displaying do not have the expected icon.

35:58 Figure out why. And I'll say pretty please because sometimes we like to be nice to LLM uh just in case. But I don't know. Uh the jury's out whether pretty please makes a big difference uh in the uh debugging process. Okay. So, uh this is a bit of a a little bit of a a preview here. Um so, this little banner that we're seeing, this is actually a sub agent that's being launched. And so, uh every now and then, uh the agent will decide, hey, you know, there's a part of

36:27 this process that I can kind of uh have another agent take care of. And, uh what it'll do is it'll launch a sub agent, which is essentially a worker, uh totally paralyzed worker that is, uh has a different context window than the default agent. and it'll have it do this sort of bucketed task, come back with the results and then use those results in the main thread. And so sub aents are also really effective for uh paralyzing tasks even within QA like let's say for

36:52 example you want to run a code coverage report uh with one sub agent and then you want to run uh you know some other documentation of a set of issues with another if you can think of a problem that's paralyzable uh sub agents are really a good choice to to do uh to do that with here.

37:08 So here we have the sub agent explore the codebase and now you can see it's uh debug mode is coming up with a set of hypotheses. So we have hypothesis 1 2 3 and four and then here it's adding instrumentation uh to validate all of these four hypotheses. So if we look at one of these changes um it's adding a post request here to actually add uh effectively like a JSON with some information to our underlying log file. And so based on this underlying log file

37:37 uh that's generated uh the debug mode agent is going to be able to figure out what the issue is. And so it added a few log lines here. And then now it's asking me to restart the front-end dev server. It is hot reloading. So I can just refresh it. Open the app, visit the homepage, and then navigate to the browse page to see the cat grid. And then scroll until we see one of those cats. And here we go. Uh this Aegon entry has an emoji. And then these are

38:03 inconsistent, too. So now it should have enough data that I've now that I've scrolled up and down and I'm going to press proceed. And so now the agent's going to analyze the log file and then uh do some reasoning to see uh which hypothesis are confirmed or denied. And we can already see by looking at the thinking trace of of the model that hypothesis 1 and hypothesis 3 are confirmed but hypothesis 2 is not.

38:32 And I imagine hypothesis 4 is also not. So it should output this soon. And then we should be able to get to the bug fix. Yeah. So here's a log analysis. H1 H3 confirmed. H2 inconclusive. Oh, I guess it was just three. And then now it is doing that lightweight bug fix I mentioned. So, it's only going to change the lines of code it deems are related to the bug.

39:07 And then we should be able to see it live. So, here we go. Now, all of our cat images have cat emojis accordingly. And then the last step here after it finishes up some cleanup tasks is to validate or to let the debug agent know that we have marked the bug as fixed. So quick mark fixed.

39:31 And now it's just going to remove all the locks. So pretty simple uh procedure, but incredibly useful. I mean uh I've definitely had bugs that have stumped me for hours and using debug mode uh helps me get through them without going completely insane, which is really nice.

39:48 All right. Uh I have one more topic I want to cover uh mainly about how to think about uh using plan mode to come up with testing plans. also thinking about things like TDD um that that are useful frames of doing agentic development. But before doing that, happy to pause again and see if there are any questions on this topic. >> One just came in um can you guide on how we can use cursor to improve application performance?

40:14 >> Um I think that's a pretty broad question. Uh my recommendation here is um to determine what metrics you're measuring in terms of application performance. So, if you're looking at like memory usage or if you have some metrics of like startup or um you know shutdown length, I I'm not I'm not quite sure. I don't really know about the the particulars of the application. However, if you have some sort of profiling software that you use um and that is accessible through a command line shell tool that cursor can

40:46 access um you can build a skill called like maybe application tuning uh and then in the prompt for the skill. So if I go like create skill uh maybe I'll call it oh oops so create skill I'll call like application JSON tuner that does a profiling of our application and suggests improvements in a report or something like that. And then this skill, I'll run it for this. I don't actually know uh uh what kind of um strategy it's going to come up with our app. It's pretty simple, but um effectively you can have this skill that

41:34 you can use over and over again as you make changes to your application and then see what the biggest uh points of friction are in in the application and then use that for further development. So uh essentially cursor you should consider it as like a very very intelligent colleague of yours that kind of knows everything about software engineering. However uh it's kind of dumb in the sense that it's not going to think for you. It's not you know going

41:58 to understand like hey you want to do this thing uh here let me proactively make all these tools for you. You just have to prompt it in a way to say hey I'm trying to accomplish this task. Can you work with me to get closer to my end goal? And in this case for application profiling or for u improving the performance guide the agent into figuring out what it is you want to measure and then finding ways to explicitly run what it is you want to measure and then using that data to actually suggest an improvement.

42:26 So here uh it's asking you know how to make a project skill. So I'm going to say store this in my project. Uh which parts of the application should my profiler cover? Let's do both. Why not? How deep should it go? Let's do full profiling. I actually don't know what this process can output, but uh we can come back to it and see if we have something cool we can run at the end. Okay, so let's move on to the last section I have here and we'll we'll come back and and loop back around for

42:53 questions. So um I want to shift gears from my uh cat stats app to a much larger uh more uh I'd say enterprisegrade application. Uh this is Graphfana. So for folks that are unaware, Graphfana is a very large open source codebase, has around 70K stars. Uh it's been around for around 10 years.

43:12 I think it has like 13K plus contributors at this point. So it's very big. Uh it's big and it's bulky and it has a lot of particularities. And so uh it is something that is very um you know uh it's not trivial to work on. And so when we're thinking about uh writing tests for large repositories that have a lot of idiosyncrasies and then have a lot of uh particulars that are hard to kind of surface uh I think what is important here is to think about testing

43:37 in terms of guard rails right so let's say uh we want to look at some underlying code that we want to change how do we make sure that we're changing that code without breaking the current functionality and so it's generally good testing practice to identify the current state write tests for that current state and then do the minimal code changes to uh optimize whatever behavior we're trying to optimize for in the underlying implementation but by maintaining the

44:03 output of the current state. So this could be relevant for you know migrating a legacy um uh Java functionality to something more performant like Go you know depending on the use case but by maintaining the input and output but just improving performance and speed and um you know uh flexibility and so for our use case here what we're going to do is we're going to um first grab a bunch of fix me and to-do tickets from the underlying codebase and then we're going

44:28 to categorize some of the top most critical um most critical items and then we're going to create a set guardrail tests for those items and then create a migration plan to use those guardrail tests and then uh implement the code in such a way that we are constantly verifying that the test pass before moving on to the next piece. So the first step here is find all of um actually you know I'm going to detail it all and use plan mode for everything.

44:54 That's that's also a good strategy here. So uh find all of my fix me and to-do tickets. Then categorize the top five most critical. Say then determine top five most critical. create a detailed testing plan to make sure the behavior is consistent and then migrate to the new functionality while maintaining nothing else breaks. So, pretty vague uh prompt, but what we'll see with plan mode is we can start with something relatively vague and we can rely on the agent to come up with a detailed implementation

45:56 plan for us. And so, even if we have kind of a vague idea of what we want to do, we can work with the agent in planning mode to kind of take something that is nebulous and and maybe uh even uh ill-formed or or has some sort of assumptions in it and we can use the agent to kind of check our assumptions and build something a bit more robust. And so the way the plan mode works is again it'll do this these same search and retrieval steps try to find the

46:18 relevant information to the problem at hand but then it'll generate a large markdown plan that you and the agent can work on before actually running uh creating any code and so it's a really good thing for uh TDD for example if you want to come up with a testing plan before writing any code just creating a plan to write a bunch of tests writing the tests and then creating another plan to build on those tests and uh implement the code that's going to uh satisfy the

46:41 the test you've written. Um either way works. I mean you can just think of it as generating a markdown rough draft or spec as an intermediary and then building off of that uh in development. And so uh it's still it's going to work a little bit to come up with this plan as we are waiting here. Let's see if our Okay, so we have a profile background script being made. So this guy's this guy's chugging away. Happy to take some more questions as we're waiting for uh plan mode to finish here.

47:14 Are we >> there's a few different requests. Um can you guide on how we can use cursor oh sorry that one was already answered. Can you walk us through an end toend example of using cursor MCP from taking a Jira story to generating test scenarios and automation scripts?

47:30 >> Yeah. Uh great question. So uh what so what you can do with the Elastian MCP so I have uh I have that here. So so once the Elastian MCP is enabled you can see all of the different functionalities that are available to you or available to the agent. And so essentially uh if you want to like look at uh some underlying details in the codebase and create a set of stories from it uh what you can do is you can have the MCP do some analysis uh of some of the underlying problems uh come up

48:04 with a plan to uh turn these into tickets and then create a set of stories for you just by firing off the create Jira ticket underlying API. And as an example here I can I can do something pretty basic to just show what this pattern looks like. So, I say based on the uh to-do comments in the alerting module.

48:34 create a set of Jira stories and opening tickets in our graphana space. And so this is going to do some lightweight analysis. In general, actually I would do this after uh the plan mode step. I think I could actually update this plan mode step to uh add a confluence dock uh as a last uh um as a last uh item here. So uh when when we look through our planning doc which we have right here uh we'll go down we'll see like there's a highle summary uh a step-by-step breakdown of all of the

49:13 things that we are or rather all of the errors that we want to uh tackle in this migration and then at the bottom we'll have a task list at the end of the task list uh I can add another uh update here to say update our uploader update our confluence dock with um details on this uh this migration plan and so effectively anything that you could do uh in Atlassian yourself by creating a story or adding a ticket. Think about how you as a human uh process that task

49:41 going from high level requirements to uh looking for information in the codebase to finally adding a ticket uh in the underlying ticketing platform. You can just outline that process to the agent, create a skill uh if if if it's a repeatable process and then just trigger that skill incursor with the requisite MCP connections and it should just do it all for you. It really is that simple.

50:03 If everything's configured, it's able to kind of coers anything in sort of a nebulous form into these standardized bucketed tasks. And so, uh, here it's running the Alassian commands. It's hanging a little bit because unfortunately Elastian is kind of unreliable. It's it's not on me. It's on it's on Elastian. But, um, uh, once it finishes, we should be able to see some Jer tickets that that'll be created. And this is pretty naive, but you can make

50:25 it as complex as you want with, uh, you know, the right amount of prompting and guidance. Okay, so here is our plan that we've generated. You can see um it's outlined all of the different uh P 0 P2P or all of the all of the different uh to-dos and fix me that we wanted to tackle. So uh the first one notific notification policy reset silently swallows errors.

50:48 Uh users think it reset succeeded when it failed. And so it outlines the actual problem. Created a testing plan here uh for what kind of unit test we want to add. And then for each of these it did the same thing. And then now uh we can, you know, we should ideally read this and truly understand it and communicate with the agent. But for the purpose of the demo, I'm just going to press build.

51:08 So once you're done with the planning phase and you've had all of your follow follow-ups with the agent, you feel confident about the plan, uh you can select the model and then click build. And this will move into the building phase which will actually write the underlying code that you've detailed in your plan. And so here I'll build with composer 1.5. I don't know if you noticed, but I used the larger Claude 46 Opus model to actually come up with the

51:29 plan and I'm using our composer model to actually build it. So, uh I like to leverage a uh a larger intelligent model, a more intelligent model to come up with the highle plan and I like to leverage the leaner faster model to do the building. Composer 1.5 is our newest uh agent model that we just released. Um it's a step above composer 1. It's very fast. It's also highly intelligent. So sometimes when I like to consider it as a leaner model, I don't think it's doing

51:55 it as much justice. Uh it is a very intelligent model and highly effective at writing code. It's also a thinking model. So it has a bit more processing power in it as well. And so you can kind of leverage uh you know heavy intelligence from these larger labs but also uh fast intelligence from uh using composer and doing both uh for planning and execution separately.

52:15 So uh we are going the agent is going through this planning step here. Um, it's probably going to take a decent amount of time to go through this. So, uh, we'll we'll go ahead and pause here and leave the rest of the three minutes for questions and then we can check up on some of these processes as they as they finish up. >> Yeah. And I see a few folks have their hands raised. Uh, Perine, Robert, if you want to come off mute and ask your question.

52:46 That also might be a mistake. Um, Rachel Ross, it looks like you have your h hand raised as well. All right, that could also be by accident. So, if folks want to come off mute and just ask additional questions, please feel free to do so. >> Oh, they can't come off mute. >> Oh, yeah, because it's webinar.

53:13 >> Gotcha. Gotcha. Well, we had one other question. Um, we have problems with cursor generating tests that silently fail. For example, it will use a reporter.log to log a failure instead of the test framework fail method to show the test as a failure. It will also wrap validation logic in if or try/catch blocks and again fail silently or bypass the validation. Other than putting this in our rules which cursor doesn't always follow, how can we solve for this?

53:44 Um, got it. Uh, do you mind Lauren? Do you mind uh pasting that uh that in the chat again so I could read it? I just want to Okay, got it. >> Okay, so we have problems with cursor generating tests that sound like fail. It will use reporter log to fail a to log a failure rather than instead of the test framework fail method to show the test as a failure. It will also wrap validation um logic if try class box and again fail silently or bypass the validation other

54:14 than putting this in our rules which cursor doesn't always follow um how got it. Yeah. So I mean the first thing I I would I would look into is making sure that your cursor rule file is very specific to the problem at hand. So um I would say uh making the rule file as lean as possible just to assure that the agent follows it. If you've had a very long chat conversation, sometimes the rule context can get overwritten uh just based on the the uh the kind of filling

54:41 up the context window. Um the other piece I I'd mention here is potentially for things like um generating tests. Uh having like a validator process or making sure that um you know you're running the test and having the agent uh look at the output of the test and then potentially rerunning it as a specific validator could be useful. uh having either that as a skill or a sub agent.

55:03 Uh I'm not quite sure at the particulars of this testing framework. But my my recommendation would be if you're doing the testing task, keep your context window as empty as possible and only have it do that task. Uh reason being is that if there's more stuff in that context window or if there's some confusion on a large set of subtasks to do, sometimes the agent can get overloaded. So if it is kind of a complex uh testing suite, uh maybe having like a specific agent working on that by opening up a new agent and just

55:34 targeting um the test running. Uh yeah, I would please feel free to like follow up for uh in like a forum post or or or share this with the community and outline the the specific issue. Uh we can probably like figure out what what the particular is and and address that asynchronously.

55:55 And then uh Povven mentions can you please publish some skills MDs files related to quality assurance. Uh we actually have um some exciting launches co coming in this department. So uh stay tuned uh we will have a way to do some really nice uh broadly sharable skills and integrations in the near future.

56:14 And also just just to cover up I think someone was like uh oh throwing Alassian under the bus. Uh I do mean throwassian under the bus. They're actually a very uh big partner of us. I think sometimes the MCV just can be a little flaky. So there's any Atlassian folks on the call. Uh didn't mean to do that. Uh big fan of Atlassian and Jira. All right. Um yeah, so thanks for everybody uh that joined the call today. Um hopefully this was useful. Uh we have many more workshops

56:42 coming in the future. We try to tackle different topics. So if there's a big need from the community to do a topic, um uh we're pretty active in our forums. Uh so please uh feel free to uh you know talk about different topics. We've done cursor for data science, cursor for PMS for design etc. So uh we'd love to stay connected. Thanks everybody. >> Thank you.