So the impetus for this idea is I'm a software developer by day and I've noticed that the current process the one overworked QA guy uses to report bugs (which, sadly, are about 99% what we work on, I can count the number of feature requests I've been involved in on one hand, but that's a separate post for another time). This process seems needlessly complicated, and only somewhat helpful for us devs.
Basically he creates a video recording and includes it with the bug ticket he created.
So, I thought, what if instead he could include a replayable user session, that devs could repeatedly run as we worked on the bug, and then he could rerun when the ticket was passed back to him for verification, confirm the bug is fixed just by rerunning the user session, and then mark the ticket fixed, all within the same tool?
So basically a browser but specifically designed for this purpose.
Other features planned: auto-detect forms and fill in with auto-generated data appropriate to the field; easily create variants of steps recorded instead of having to record separate user sessions.
User sessions will be JSON files containing intelligently chosen DOM selectors, and event information (i.e. for a form field there's usually no good reason to record click x,y). The intent here is to make the user sessions as robust as feasible so if, for ex. an input field is moved somewhere else on the page but retains its 'id', the user session replay is unaffected. If elements can't be located during a replay, instead of session failing it will pause and prompt for tester to perform replacement action, and then continue with rest of replay.
A companion tool is also planned to read these user session files and execute them using selenium, webdriverio, etc. for cross-browser headless testing.
I've already coded a very simple bare bones app (turns out using JavaFX WebView this isn't even that difficult), but having read posts on here about the importance of validating first I'm wondering:
Such tools exist in various incarnations: Capture for Jira, Usersnap, Zephyr.
None of those is a fully-fledged browser automation remote control solution like the one you described, though.
If you decide to go down the route of implementing such a tool I'd suggest looking into existing tools such as Puppeteer, Cypress, or the tried-and-true Selenium. RPA (robotic process automation) tools might also be worth looking into.
However, doing so is no small task and creating such a tool probably requires a huge upfront investment before you'd even be able to validate its value proposition. Querying the DOM in a robust, scalable, and automated manner is a hugely complex task.
Perhaps, I'd look into small parts of this QA process or subtasks that might lend themselves to being optimised first. With Jira the problem often lies with its configurability. This can lead to badly designed processes with lots of statuses and required fields that mostly cater to the needs of management and managerial accounting, specifically, rather than the needs of those using Jira for everyday work.
Streamlining this process via a plugin or a process template for example might provide a more readily available remedy to this particular pain point.
Thanks for the detailed commentary, highly appreciated. As I researched it seems that most of these tools (Capture, Usersnap, etc.) record video/screenshots of user interaction. This is helpful to a point, but it's not great. For one, you can't rerun the recording to verify the issue is fixed. For two, you can't pause and inspect elements, etc. since it's just a video recording. Perhaps though that's where a big part of my focus needs to be: making it clear the limitations of video recording and the significant advantages of having true user session recordings.
There are some tools that do already exist that seem close to what I'm doing (LogRocket seems to make actual user session recordings not just video recordings) so I'll definitely be doing more research to see how I can create a unique value proposition from them.