However, I think when we’re learning, it’s good to have examples we can refer to – that’s always the shortcut I use to get inspired anyway! So here are some working examples I created to help get those creative juices flowing. Note these are all basic Flows that may form part of a larger Flow should you wish to do something more advanced.
All of my Flows use the Restful Booker API, written and maintained by the amazing Mark Winteringham. Click on the links below to access a video and short tutorial on each Flow.
In this instance, we create a new booking, and pass the created booking ID to a request to GET the information about that ID. This is done via the the “assign values to variables” link within a request without the need to use the now deprecated Assign Variables/Create Variables blocks.
Video showing use of “Assign values to variables” link to define and pass variables – remember to create an example first!
Flow #2: Looping data
*Updated May 2022* If you want to loop through a response and apply a rule to it then here is an example. Here I take the ID’s from a GET all request, loop through them using a FOR EACH loop and for each of the ID’s in the request I output to a terminal if the booking id is greater than 100.
For Each Loop Highlighted in Green
Another way to do this is via the Create Data/Loop Over List Flow, also shown above. Using these blocks, I also execute a further request using the ID from the loop e.g. to GET information on the booking. I then output the test run results to a Test Summary block.
The steps are as follows:-
Ensure you have an example saved in your collection for the requests you wish to use in the Flow
Add a Send Request block
Add a Create Data block, using the data input to select a value (in this case /data/body/ which is a list of booking ID’s)
add a Loop Data Until block, which tells the Flow you want to do something with the data you’ve created
Add further blocks to do that thing e.g. Send Request (adding the variable of bookingid you’ve generated above) or conditional logic
(optional) to convert the data back to its original state, add an End Loop Block
Add Terminal/Test Summary Blocks to output the data so you can ensure it works – it will dynamically update with new information each time the data passes through the loop
When Looping, you must first Select the data you wish to loop through
Flow #3: Take Inputs From Multiple Sources
This example shows how you can take requests running against two different environments from two different collections and perform a Check on them both to output to a terminal.
Video showing outputs from two API requests flow into the same check block in Postman Flows
Flow #4: Passing Data Between Requests
From version 9.5.0, you can use the Create Durables block to complete the task of passing data between requests – as long as you have created an example API request first! By adding the token as a durable type, the generated value can be persisted throughout the Flow.
Video showing data being passed through a Postman Flow using Data Durables – Remember to create an example first!
Flow #5: Generating a Test Summary Report
My fave Flow! Takes tests run as part of 3 different requests and outputs the results to a single Test Summary Block. Note these can be from entirely different collections if, for example, you want to generate a quick eyeball test report from across all your critical APIs.
TL:DR – see below section for a list of useful resources to begin learning more about Application Programming Interface (API) testing.
Today I got a message from Coders Guild software testing graduate Alexander Olakunle, who I have to say I found to be a very dedicated and driven person, who has worked incredibly hard on the course and landed his first testing job with Ordinance Survey (congrats again Alexander!). He asks:-
During the short training course, we cover API testing, but only from a high, theoretical level.
I thought it might be helpful to others (that means you!) to list some excellent, beginner-friendly resources to try out if you’re in a similar position to Alexander.
Talks
If you like seeing how different people tackle the same problem, I’d definitely recommend taking a look at the Test.Bash(online) talks from the Ministry of Testing. 4 different sets of people answered the same challenge, and gave demo’s on how they did it – so many useful tidbits of practical insight there.
Dave Westerveld has created a great book (published by Pakt) called “API Testing and Development with Postman” which serves as a fantastic introductory book for API testing (don’t let the development bit of the title put you off!).
If you complete that course, you can then go on to do my Test Automation Course API Test Automation with Postman, particularly if you’ll be using the Postman tool for your testing efforts. If REST Assured is something you’ll be using, Bas Djikstra has a TAU course on that too.
Certifications
I’ve only very recently become aware of this, but there is an organisation called API United, who have created an API testing certificate. It currently costs £170 to take the exam in the UK, but the syllabus is available online so its possible that this would be all it costs you to get the API-U Certified API, Rest & Microservices Tester (CARMT) with Postman or Karate
Screenshot showing the certification title and API United logo
Of course certification isn’t for everyone, but if you feel it would benefit you it might be worth taking a look.
More Advanced Stuff
Lewis Prescott – A master of all things contract testing, anything he appears in is a great resource if this is an area of API testing you want to learn more about, especially his podcast series “How to start API Contract Testing“
Mark Winteringham – Not a lot of people know this 😉 but Mark is gearing up to publish the final version of his incredible book Testing Web APIs. Its got loads of practical exercises in, so if you enjoy working through those you will get so much out of reading this book.
Of course, its well worth doing your own research, and I’d love to hear if you have found other resources elsewhere that you’ve benefitted from – drop me a comment and let me know, that way I could learn something new too!
For the last two months, I have been working as interim QA Relations at Mailinator. It isn’t often in our careers that we’re able to have a risk free go at trying something new. But wanting a bit of a gap from leaving my full time testing role to starting my next one, this opportunity was one I had to grasp.
Would I like to do QA/Dev Rel type stuff in future? Was it what I expected it to be? Would I recommend it to others? Read on to find out my honest opinions…
What did QA Rel look like?
Despite being almost 20 years old, Mailinator had never had a QA Rel before. Therefore, this was a great opportunity to evolve and learn from each other as we went along. Alongside weekly meetings with both the wider company and individual 1:1’s with the CEO, I was given a lot of free reign and some loose guidance along the following lines:-
Get access to social logins and regularly post interesting stuff
Learn more about the product itself
Write a blog post or two for the site
Give your professional insight as a tester on how to improve the service
Having a team that were enthusiastic and super keen to hear the views of a tester (their key customer) gave me confidence to give my opinion on a lot more areas than I first thought I would. The following things give a flavour of the key activities I got involved in:-
Social Media Stuff
What I did
Tweeted and Posted on LinkedIn and Twitter most days of the week
Polled users to find out what freebies they prefer
Created 6 tutorial video’s for an Essential Guide blog post
Penned 4 further blog posts
Learned how to use Yoast to improve a posts Readability and SEO scoring
screenshot of Mailinators blog page – all blogs shown were penned by myself
What I learned
Creating a blog for a company is a different ballgame to creating one for yourself. I thought this would be the easiest part of the role, but it was one of the hardest. I learned to give myself plenty of time both to write the initial draft of the blog, improve the copy using the plugin Yoast to get good SEO, accessibility and readability scores. I also took the extra time to insist on getting all posts reviewed by the team. The feedback definitely helped make the blogs better quality and avoid a few clangers!
Also – people really do not like to follow companies on Twitter and LinkedIn. I found it much harder to get “engagement” through the Mailinator account than when I posted as myself.
Testing Stuff
What I did
Created a Trello board of new bugs, customer insight and recommendations
This wasn’t part of the brief, but it is actually impossible to spot something that could be better and not feel compelled to be a part of making that happen. My Trello board was shared with the team, and included a lot of annotated screenshots (I use Techsmith’s Snagit – a paid tool but worth every penny).
What I learned
Not everybody loves Trello! There were a few bugs that made their way through the system and were reported by users that I had on the Trello board beforehand. I guess the lesson there is to find better ways to highlight important bugs and find a way to communicate that works for the whole team.
Reviewing Existing, Revealing New Features
What I did
Created Mailinator’s first public API in Postman, including a “run in Postman” button to easily get up and running.
Tweet from Kin Lane, API Evangelist, praising the Postman Mailinator Public API Workspace
Gave insight and feedback on a new pricing strategy and other features
Completed a competitor analysis
Created a document identifying 10 ways the company could improve that I debriefed with the CEO
What I learned
This was mostly stuff I hadn’t done before, so trying out new things and getting positive feedback was really rewarding. Sometimes its worth pushing yourself – as long as you’re not BS-ing about your experience beforehand, everything is a learning curve!
Also, at my leaving meeting the lovely CEO Paul said that having someone on board the team that was so enthusiastic about the product being a sounding board for new features meant that there was stuff now baked in to Mailinator that otherwise wouldn’t have been. That’s something I didn’t see coming, but a nice impact.
Also, I love Postman. Tell me something I don’t know. 😉
Working with 3rd Parties
What I did
Lined up guest bloggers to continue to provide content in the coming months
Coffee with Simon Tomes, community boss at MoT, to talk all things community and ask advice
Asked the community for a recommendation and brought on an amazing Tech Writer to improve documentation
Liaised with Ministry of Testing Marketing, Sales and Community peeps to understand how they could help Mailinator to improve their interactions with the testing community
Arranged for Mailinator to Sponsor a MoT event
Liaised with the community and Postman DevRel Sean to further improve the public API collection
Worked with the lovely Himanshu at cloud based cross browser testing provider Lambdatest on exploring a possible integration. This involved investigating Lambdatest’s capabilities, asking others in the community what they thought of Lambdatest, and doing a proof of concept to understand how any integration would work. Have to say I was pleasantly surprised by what this platform could do and how positive others were about it – so much so that I used it to do a tech test when applying for a new role.
What I learned
I learned that speaking to other people and forming genuine and positive relationships can be done, and was much easier for me than I thought it would be. (trumpet well and truly blown there Beth!) I made sure at the start of the assignment with Mailinator that QA Rel was not going to be another word for Sales – an area which I’d be terrible at and which they 100% agreed with – and I definitely think that is an important principle to stick to.
At least 2 of the above 3rd parties have reached out to me to ask if I’d be interested in doing further work for their organisation, so this feels like something I did a pretty good job at.
Metrics
What did I do?
Getting before and after data on our social media activity wasn’t part of the role, but I felt like it should be. If for no other reason than to measure my own success.
Increased number of Twitter Followers by 112%
Increased number of LinkedIn Followers by 554%
Increased visitor metrics to Linkedin page by up to 850% (average 200%)
We aren’t talking about huge numbers here – for example we crossed the 1000% followers mark on Twitter a few days before I left. But given Mailinator had been on Twitter for 12 years, I think this was a pretty good milestone!
I learned that all of this stuff isn’t really about numbers. I’d sooner have one meaningful conversation with someone than 100 likes to a post. But, over time, they do give trends and they do tell a story. Focus on the macro side of things.
Would you recommend QA/Dev Rel?
In much the same way that since home schooling last year has made me appreciate the teaching profession in a whole new light, I now feel the same way about Developer / QA Relations.
Dipping my toe into this stuff for, lets face it, a very short amount of time, has really made me realise what bloody hard work this stuff is! There is a world of difference between doing this stuff for yourself, and being paid to do it for someone else. It takes a lot more energy than you’d think, even for small stuff like writing a tweet in just the right way.
I think I enjoy talking about products, tools and companies that I genuinely am already passionate about – and Mailinator 100% fitted that category. This made things so much easier, because I didn’t have to fake authenticity.
Would I recommend it? I think perhaps as a side hustle its a fun thing to do. And I’ve met some mega people that I’ll definitely be staying in touch with and hopefully will help out in future. I hope for myself and other testers, I have done a good job at persuading the Mailinator team of the importance of decent swag. But it absolutely made me realise that I missed testing too much, and after working in QA for 15 years, that’s definitely where I want to be. I’m so lucky I had the chance to try this out, and am so grateful to the incredible team at Mailinator for helping make it such a blast.
I start my next adventure tomorrow, with Ada Health, as a Senior Quality Engineer.
LinkedIn Post announcing I am starting a new position
I’m all about the stories, so if you’re reading this because this type of thing is something you’re considering, please get in touch via the contact form or LinkedIn and tell me your tale!
I’m now going to publish this post with no review, second opinion or Yoast. So its back to mediocre blog posts I’m afraid folks, but I think for me, that’s exactly how I like it 😉.
Do you want to learn more about using Postman’s feature Flows, but are stuck for inspiration?
Want to see how it looks and feels in the latest version released just last week (late Jan 2021)?
Then have a gander at the livestream I did with the Postman team. In the video below, myself, Arlemi (Senior Dev Relations at Postman) and Vikram (Developer Advocate at Postman) :-
Create 3 basic Flows from scratch 🎓
Talk about how Flows has evolved and show off the latest improvements 😎
Let the cat out of the bag to say when flows might come out of Beta and into general release 🤫
Video of livestream of Postman Flows feature
And if you want to read more, take a look at my earlier posts on Flows here:-
I’m very *very* excited to finally be able to say it out loud. As you may have heard, I’ve now left my previous role as Staff Quality and Test Engineer at Smoothwall.
I’m genuinely thrilled to announce that for the next 2 months I’m going to be helping the team at Michigan based email testing specialists Mailinator as their QA Relations advocate. 🎉
I’ve known CEO Paul for awhile and when I mentioned I was taking a few months off he reached out to see if we could work together – and there was no way I was going to turn this opportunity down. ✨
I’ve since met the Mailinator team and can confirm they’re a lovey bunch, their product is legit amazing (I couldn’t back it in all good conscience if it wasn’t) and I’m itching to get stuck in!
Who are Mailinator?
I’m going to be writing a lot more about Mailinator over the coming weeks, but here are some of the systems features I’ll hopefully get to show off.
Key Features of Mailinator
Ever got stuck needing to automate a signup process that asks you to click a confirmation link sent via email? Or maybe your end to end tests fail because you’re asked for 2FA verification when logging in and you can’t automate that? Or how about testing the SMS messages your system sends? If so, I’m here to show you how Mailinator can help to fix those problems.
So many things to talk about, just hope I have time to do it before I plan on returning to the world of testing in March.
So yeah, if you can’t tell, I’m just a tad excited about this particular gig. Please reach out to me on Twitter or LI if you want to hear more – note I’m not a sales person, my job is to start conversations and share my knowledge. Let the adventure begin!
TL:DR – below is a post I wrote when leaving my last role as Staff Quality & Test Engineer. In a very strong market with lots of people reaching out to work with me, this helped me to put my finger on exactly *what* I wanted to get from my next role, and meant I didn’t persue opportunities that weren’t the right fit for me. It also meant I could privately share this with recruiters or hiring managers when they asked “So, what is it that you’re looking for?” – Having an answer to this question in advanced definitely paid off. Feel free to use this template or create your own when contemplating your next career move. 👍
———————————
Inspired by Elisabeth Fiennes’ wonderfully written article all about QA recruitment, I decided I would take her advice and use this space to tell you, potential new boss, what I am looking for in my next role. This is written in the hopes it will help you make a quick assessment as to whether I am someone who you could see yourself working with. If I’m not, no harm done, and no time wasted. If you like what you hear, you know where I am!
What have I done so far?
I’ve been testing for 15 years now. I’ve been a contractor, consultant and permie, and I’ve enjoyed them all. I’ve worked for some of the biggest companies in the world, and some of the smallest. I’ve also worked with some incredible people. But I have to be honest, despite successful delivery, repeat work, glowing recommendations and even poaching requests from clients, I’ve suffered from imposter syndrome at them all.
The pandemic was a huge leap forward for me in terms of what I have pushed myself to achieve outside of work. I’m so proud to be a Test Automation University instructor, as well as to have helped many people more locally land their first testing role following the Coders Guild bootcamp I helped to create and deliver, including designing and delivering a full day workshop on API testing.
I like helping people, but I also like to be helped myself, and there is so much still to learn – you never really master everything I know, but I want my growth curve to be steeper than it is currently.
Within work I’ve been promoted to the only Staff Quality and Test Engineer in my company, I’ve persuaded my company to hire and taken responsibility for successfully onboarding an ex-bootcamp learner as their first ever Junior QA. I’ve rewritten a training programme delivered through the Web app I currently work on, which has certified over 25,000 teachers, dinner ladies and school bus drivers up and down the UK. And lots of other things I’d love to discuss over a nice friendly interview chat. <nudge nudge >
What are my career goals?
I really want to grow my technical skills. I know everyone says that, but I feel that I’ve done as much as I can to grow them on my own and I need to work in an environment that actively supports me to improve areas such as API exploratory testing, Shift left and right practices, non-functional, DevOps, code reviews, accessibility, security, automation etc etc. I learn best by sitting down next to someone approachable and kind, getting feedback on my work and measuring progress. I want to be confident in my technical ability.
I want to speak at my first international face to face testing conference and I think I am on a good trajectory to making this happen.
I want to achieve something meaningful. I like being able to say I’ve made a difference.
What kind of company culture do I enjoy best?
Here’s a wee list, I expect my next company will have or be working towards most, but probably not all of the following:-
A quality culture which enjoys finding out new things, experimenting and promoting quality. A team of people who want to do the right thing.
An experienced and technically gifted individual or set of people who can coach me. I want to seriously assess and improve my technical skills with hands on code reviews, pairing and training – if your workplace can’t commit to 1 or 2 hours a week to support me in this effort, then it has to be a hard no from me.
Vision – A leader with a vision I can align with. I’ve been most successful and found work most enjoyable when I can hook onto the vision of a leader and focus my energy on implementing what they want. I’m great for bouncing ideas off of, but really I’m a completer finisher rather than a big picture person.
Remote – Hybrid working is the model for me – I am happy to work towards being fully remote, or 1-2 day per week in the office (within West Yorkshire). If a monthly in person is required, I can stretch to Manchester, York, London or Newcastle. If its quarterly, I can travel anywhere in the UK and once or twice a year I can travel anywhere in the world. I can drive if necessary but I also live close to train and bus options.
Flexibility – I currently work 4 days a week, the 5th day being a flexible choice. I am very flexible to the companies needs, and often work evenings or have early starts in order to get the job done (as well as the occasional weekend). I’m focused on delivery rather than presenteeism and would expect my workplace to share the same values.
I hope you are still reading at this point, and think my expectations are realistic. I hope there are plenty of companies out there who can meet them. Fingers crossed you are one!
In return, you will get a motivated, enthusiastic and passionate quality professional who will put business needs in front of ego. Someone who is a proud, committed and connected member of the global testing community. Someone who will always aim to help your team and company gel, improve and ultimately succeed.
Postman have recently released their huge API report. It gives a long term view on the world of API’s, and includes insights from all Postman users – incredibly over 26,000 of them who responded.
I have previously blogged about a few similar reports, and what they tell me in my role as a software tester:-
The Postman report was interesting, because it was aimed at all users of API’s – the majority of whom are developers. You can read the report in full here: https://www.postman.com/state-of-api/#key-findings
Here are my three takeaways:-
Takeaway #1: Postman isn’t just for ad hoc manual testing anymore
Extract from Postman’s State of API Report 2021
Whilst a lot of people do still use Postman for ad hoc/exploratory testing – and as Amber Race’s seminal Test Automation University course shows, it is brilliant for that – its now clear that automating API’s isn’t seen as an edge case or extreme thing anymore. Among all respondents, 11% of time was spent on automation, but amongst Quality Engineers it was a whopping 31%. This also adds weight to the assumption that testers are responsible for automated testing in the majority of organisations, something which the other reports (mentioned above) seem to back up.
Takeaway #2: Its a man’s world
I’ll just leave this here….
Extract from Postman’s State of API Report 2021 showing 86% man, 10% woman, 1% non binary, 3% prefer not to answer
Extract from Stack Overflow Developer Survey, showing 91.67% man, 5.31% woman, 1.75 prefer not to say, 1.42% non binary and 0.92 in your own words
There is clearly an issue here which is tech-wide, and bigger than one company to fix. But in fairness, I think Postman do a great job at encouraging female voices to be heard, and I look forward to seeing how they seek to chip away at this imbalance in future. (*whispers* It’s actually my next career ambition to present f2f at a non-UK conference, maybe Postman Galaxy would be a good way to pop that particular cherry? 🧐#justPuttinThatOutToTheUniverse)
Takeaway #3: Remote Working Isn’t Going Anywhere
Extract from Postman State of API Report
This is one of those responses that asks more questions than it answers – I’m intrigued! It’s really interesting to me, because with 26000 replies its probably one of the most accurate bellwethers we have for tech as a whole in relation to this issue – most companies have API’s now, so its safe to assume the companies we work for fall into these categories.
I guess I would have liked to reverse the question to find out if the respondents expected to go into the office every day or at least one or more days per week (as opposed to working from home during that time). There is a big difference between being fully remote and going into the office 4 out of 5 days. It wasn’t the focus of the survey, but I’d also love to know how keen the respondents were to go into the office/work from home – my personal experience is that the younger (and often more inexperienced) folks prefer the occasional office day as its easier for them to pick up things, but its a jump to assume that’s what these Gen Z-ers thought.
In Summary
I’d recommend taking a squiz through the survey yourself – it is super easy to absorb (props to whoever put those graphics together). Its pretty clear that API’s aren’t going anywhere, are truly a global thing, and that Postman is by far the most popular tool in the market.
tweet from Jim Evans announcing Selenium 4.0 Release
What does this mean for me?
According to the recent Mabl state of devOps report, Selenium is the tool of choice for the overwhelming majority of survey respondents using a functional testing tool:-
Extract from Mabl’s 2021 State of Testing In DevOps survey showing 41% of respondents used Selenium
If you already have Selenium and are wondering how easy it is to upgrade – the answer is, pretty easy really. Don’t be scared of it. It makes sure you’re using the latest code, plus there are a bunch of cool new features in this version for you to try, so upgrading makes sense.
That being said, I wanted to point out a few things that caught me out while attempting to upgrade from Java Selenium 3.1.14 to 4.0 today.
Where are the best instructions on how to upgrade?
One thing that perhaps is obvious to people who play with this stuff all day but wasn’t to me, was the missing reference to pom.xml. Perhaps because other setups use a different filename to hold their Selenium dependencies the articles weren’t explicit, but as a Maven user, in order to upgrade to Selenium 4.0 you need to go to the pom.xml and find the Selenium Java dependency and update the version like this:-
step 1: open the pom.xml (maven users only)Step 2. change version
After you save this file, you may see some errors. These are visible in Eclipse as problems, or will appear if you run “mvn clean compile” from cmd line from the same folder as your pom.xml.
The selenium instructions include some more detail on this if you get stuck, but my errors were all to do with Desired Capabilities. From what I understand, this is essentially deprecated in Selenium 4, because it isn’t W3C compliant.
To fix this, we need to use browser specific Options instead.
From this:-
To this:-
I had to replace all instances in my code that referred to the previous DesiredCapabilities. Once this happened and I saved the file, everything went red. 🤯🤯🤯
I panicked. I thought I’d broken EVERYTHING. I sought advice from a more experienced automation guru than me at this point, and the fix was simple.
To fix this, you simply needed to run a command to update the Maven Project. In Eclipse, the keyboard shortcut for this is Alt+F5 (although I’m guessing the cmd line “mvn clean compile” does the same thing.)
Hurrah! That was all it took! Selenium 4.0 is now enabled 😊.
I’m sure these instructions won’t be applicable to everyone, but if they have helped you at all then that’s cool.
“We’ve done a soft launch for a new feature called Flows, which we’ve been working on for ages. Basically it is a widget based canvas that you can drag and drop components and link them together.
Previously if you wanted to start chaining requests at Postman, or start carrying values between requests you had to know a fair amount of how to structure JavaScript and work with code.
Flows is basically our codeless approach to chaining processes together. It really opens up Postman to a whole new audience.”
Now I have been waiting for this to drop for awhile, partly out of sheer consumer curiosity as someone who is still reasonably inexpert when it comes to code, and partly because I had originally wanted to include a chapter on this feature in my Test Automation University Postman course. I think any technology that wants to make it easier for non-tech people to get into and not harder gets my vote, so this is an exciting area for Postman to move into.
I’ve had an hour or so to take a look at the beta, and here are my thoughts, as well as tips on how to use it.
How do I find Flows?
Flows can be found in both the Web version and the desktop Postman app.
To access Flows:-
Select a workspace
Select Flows from the left hand horizontal menu (underneath History)
Click on “new flow” to view the canvas
Click on the “flow” menu on the bottom of the left hand side of the screen.
Where to find Postman Flows
How Do I Get Started Using Flows?
It is very important to say that this is only a BETA version, and inevitably lots of changes will happen between now and the final formal release. I have to be completely honest here and admit that it wasn’t the most intuitive feature for me to get to grips with, but nor was it so painful I had to give up on it completely. I’m sure by the time you come to read this Postman will be several iterations in and it will look and feel much better.
If you wish to explore for yourself, the Flows work by adding blocks in a chain. Think like a daisy chain necklace. Currently these blocks can take several forms in Postman:-
video showing current list of blocks in Postman Flow
There are tonnes of use cases for this that I’m sure your brain is whirring with already, but I’m going to start with a basic example.
Say you are using an endpoint that returns a list of bookings for your hotel. You want to see if there have been lots of bookings, or its been a slow day and you haven’t had much success.
With PostmanFlows you can use blocks like “send request”, and then add a chain to a “for each” condition – this allows you to iterate through the response and run a query against it using a third block called “validate”. Finally, as the validate outputs a true or false data, each of those records can be sent to a different “terminal” log group, allowing you to view what data has been applied to what “pot”.
Picture showing an example Postman Flow
You can see this for yourself in much more detail in this vid:-
How to use Postman Flows
*NEW* And here is another vid on how to chain requests together to “flow” data from one request to the next. (updated March 2022 to show the latest functionality of Postman Flows version 9.15).
*NEW* How to pass data using Assign Variables and Data Durables
I hope this very basic guide to this new feature is useful to you. I’m looking forward to what future iterations bring, and will aim to update this blog post with future updates.
A few months back, I blogged about a testing report which aims to cast light on the state of testing. That blog was weirdly popular. So much so that when I saw Mabl had released a similar report, this time focussing on DevOps in relation to testing, I thought it would be remiss of me not to talk about it (love that phrase!).
So, what is the Mabl report all about? This is the third year running that the popular automation tool has surveyed those in and around the QA profession about how, or indeed if, they work with DevOps. 600 people were surveyed, and there are some great insights. You can find the detail here, curtesy of Software Testing Weekly.
Now, if I’m letting my imposter syndrome hat drive my pre-report thinking, I’d have been expecting it to paint a picture of pretty much everybody being fully DevOps. Think multiple deployments a day, 100% automation at all levels of the stack, perfect pipelines and testers (if they even exist anymore right?) scratching their heads thinking “what can we do now?”.
However, as with so many things, the reality lives down to the hype.
Some of the facts that struck me were:-
only 11% of those surveyed classed themselves as “fully DevOps & Automated”. The majority were Aspiring, or Striding (in order words, not there yet).
there was a 14% decline year-on-year with those who said they had adopted “CI” (continuous integration) principles. 33% said they were in the process of “Transitioning” to utilising CI.
Just 27% of respondents said they had adopted Continuous Deployment.
How about regular deployments? Almost half of teams still deploy new code less than once a month – so if you’re in a team where that is happening trust me you are not alone. Just 15% deploy daily or more, and this %age has actually decreased year-on-year.
When it comes to what layer of the stack to automate, unit, UI and regression are the firm favourites – but all of those have only around 40% of teams saying they automate them.
Perhaps the key one for me was this.
One very interesting revelation from this year’s data is that most teams – regardless of where they are in the DevOps maturity – rely on a combined automated and manual testing approach.
Mabl, State of Testing in DevOps Report 2021, page 17.
We often here “manual testing is dead” type themes from those who want to emphasize the silver bullet type benefits that automation can supposedly bring. I’m with Melissa Fisher here, click this to listen to her insightful Racket on this topic. Automation is a useful tool that can help testers, but it can’t replace them, and nor should it try to. Speed should not come at the expense of quality.
Whilst the companies who are nailing DevOps are more likely to have a culture of quality, and even though they are smaller in number than we might have realized, they will undoubtedly be destination companies for testers keen to embrace future patterns of working. Customers of these companies seem to be happier too, which is a great measure of success in my book.
So DevOps does appear to be on the horizon for a lot of testers. We’re just not all there yet.
It won’t be an overnight move, and if you happen to work somewhere which hasn’t fully embraced DevOps ways of working, whether that’s due to cost, politics or a plethora of other reasons, you don’t need to consider yourself over the hill and unemployable. If you’re somewhere that does bits of DevOps, but is still trying to slowly do more, you’re not alone.
And if you think you have to only do 100% automation to survive in the world of testing, you might want to think again.
Beth The Tester's Tales 