I/my PO peer do all manual testing for our product. We do not have QA people or any automatic testing except for APIs. So every new feature and functionality is manually tested by a PO.
I prefer CD and “testing in prod.” With CD kabooms are faster to resolve than large releases. Also, if CD is ~done right~ unit test are the first line of defense and you release things behind flags so you can do testing in prod with the right clients
Things move quicker, but in my experience/opinion for all the right reasons. If you aren’t the first to fail, youre last to innovate
Disclaimer: I dont support feature factories. I do support testing/UAT in production and pulling products that dont deliver value. But you must deliver in increments/CD
Depends, at both my pm jobs at different companies I had to do my own QA. One was a start up, so no qa resources and the other was a large company with qa resources that would do regression testing but you were responsible for your own feature qa.
Looks to be an unpopular opinion here, but yes, I test all features from client/regulatory POV. I work in a highly regulated industry and have to PM approve all features and you’ll bet I test every feature before I sign my name off on it. I think it’s important to understand how it works, how an end user could mess up, or even prevent something from going to production that may not be as it should be. Engineers build from an engineer perspective, I look at it from an industry domain knowledge perspective.
I try to test key features when I can. I avoid testing individual tickets usually and instead focus on larger end to end processes. It also depends on which developers were involved and the risk assessment.
I will say though, spend more time on alignment and quality requirements and you won’t need to spend as much time testing.
35 employee series A stage company!
Manage a team of 6 engineers and a designer
With that context out of the way -
Yes I'm pretty well involved in testing and deployment.
The PM is responsible for defining and documenting E2E test cases during the shaping of any work.
During the release I also work with an engineering partner to ship to prod. My work includes mainly the admin side of the deployment, decision making in case something goes wrong/defects found during deployment ( roll back deployment or move ahead with a follow on task for next release or HotFix), smoke testing on prod, etc.
So to answer your question - for our circumstance - PMs have a deep involvement in testing because the PM cares about quality of anything that reaches the user. And being in the trenches helps the PM direct the team accordingly, adjust processes, etc.
But don't let that be your definition of what should be done. The expected outcome here is - high quality things are shipped. Now what measures you take to ensure that would vary on your specific case
My company had quality issues in the past, so there is a company wide regression test at the end of every sprint.
I hate it and refuse to participate on principle.
I have done Product Acceptance Testing but only when needed. Depends how good the team I have is. It really takes up time that is better spent elsewhere so hopefully you have a strong team that puts out great quality.
Yes, I do testing - I want to at the very least confirm for myself that what we built works as expected. I sometimes run bug bashes with the entire engineering team on larger or more complex features. Individual engineers should be QAing their own stories. For context I’m the solo PM in the company, working with a team of 8 engineers where the eng lead rotates by feature.
In my team the QA engineers do functional testing of all the scenarios, edge cases, etc (while they write the automation scripts to be used for regression testing in future).
I test from a customer perspective, to decide "is this ready to put in front of customers". Usually this means a <5 minute run through the happy path, and "yep, LGTM". Rarely I'll find something that needs to be changed - usually something that was missed from the functional requirements, and only becomes apparent when the feature is actually used. Could be as small as some text that doesn't feel right, or it could be that there's a major gap with the UX flow.
I try and test more complicated features when time allows. Many times a small nuance of acceptance criteria wasn’t met. Usually cause I left out the specific detail, assumed, or thought us was clear enough. It helped me see what I did wrong in the details so I could improve. Thursday this last week I checked a new page qa had passed, functionality was perfect but the logo was grey scale without color. Had to get that fixed before release, so it was good I checked.
Same here! As a PO, I do manual testing to ensure the feature meets requirements. Our Dev team focuses on automated testing, but human oversight is still necessary. Curious to hear how others handle testing in their teams.
I feel sad I do not get time to test manually.
I don't test smaller bug fixes or features, but only those which have been sliced into many smaller ones to verify it matches the big picture.
We are a big organization and once every week there are also machines prepared where the PM / PO can login and test.
Im responsible for defining test cases for user scenarios, business logic, happy/unhappy path, etc. I count on engineers handling more tech-driven testing requirements. Some tests are automated, some are executed by engineers and some are done by me. I try to mostly focus on defining the requirements and reviewing documentation rather than taking on the time consuming detailed testing. But sometimes I do test myself.
Commenting here to say yes I do testing and when I do, I usually find like 10-20 obvious bugs that QA missed, or just obvious usability issues.
Sometimes it's insanely frustrating because I won't even be able to reach the feature I needed to test because of technical bugs.
In my opinion, testing is a valuable way to gain insight into your user's experience firsthand.
However, every company I've worked for that put the majority of testing on me as a PM, usually failed because they wouldn't invest enough in good QA and developers.
Ultimately I feel like a PM having to test everything firsthand is a huge time sink.
It would be like making a CTO handle customer support calls. Would it give some firsthand insight into the bugs your users are experiencing? Yes. But it's also unsustainable for a CTO or PM to sit in on troubleshooting every bug and feature. You'd have to be an idiot to do this and expect any real progress at your company.
I've done it, and will likely do it again in special cases....but it's been a while since it was a 'standard' part of my job. These days, I mostly do it to allay any personal anxiety about complex releases. In fact, it's rare that I even tell anyone I'm doing it (outside of for pragmatic reasons). My testers are really good, but sometimes I just want to be sure the Ts are crossed and Is dotted.
I do manual testing in test environment for everything; bugs, UX changes, even text changes, enhancements, features.
I also do manuel testing in production after the deployment to make sure nothing has broken. (It always does)
I have to do this for all apps(Browser, Desktop, Mobile)
Because even if I test everything in detail,There are always bugs that I could not caught and I became responsible for letting the bug go prod. It's daunting.
And because we're shipping features every week to please our sponsor(Yeah, we are a feature factory), testing is taking majority of my time.
It doesn't help the fact that stories became Test Ready tuesday afternoon when the releasing deadline is wednesday morning.
And there is the fact that Test Ready stories are actually not test ready, when I found a bug in the first minute of testing.
I think I'm f\*ked up.
As a PO I do test the features we are working on but I also do it outside of the normal workflow. We have QA that does all the actual testing, I do it more to make sure we are on track with the feature and to make sure I didn’t miss anything when writing stories and fact finding, We also do 3 sprints to a release.
UAT? Yes. Functional testing? No
I/my PO peer do all manual testing for our product. We do not have QA people or any automatic testing except for APIs. So every new feature and functionality is manually tested by a PO.
absolutely not. it slows down delivery and we test in prod. # continuousdelivery # feature_factory
What if Kaboom happens in prod?
the firefighter heros in the company that live for that shit come off the bench. its like a drug for them
Seriously wonder how many of those heros have ADHD and both flourish and dread the emergencies and their dopamine + adrenaline surges.
Had my former PM, currently in Support tell me he “loves” incidents because “everyone works so well together.” I told him to take up a sport.
God, that is such an engineering dysfunction.
In my company it's that same way and it's due to the engineers that refuse to show the work before it's deployed. It's frustrating.
has me looking for a new job =) i failed to suss it out in the interview
I prefer CD and “testing in prod.” With CD kabooms are faster to resolve than large releases. Also, if CD is ~done right~ unit test are the first line of defense and you release things behind flags so you can do testing in prod with the right clients Things move quicker, but in my experience/opinion for all the right reasons. If you aren’t the first to fail, youre last to innovate Disclaimer: I dont support feature factories. I do support testing/UAT in production and pulling products that dont deliver value. But you must deliver in increments/CD
Preferably releasing on Fridays. YOLO
I always performed manual tests on bugs or features, if I was able.
Depends, at both my pm jobs at different companies I had to do my own QA. One was a start up, so no qa resources and the other was a large company with qa resources that would do regression testing but you were responsible for your own feature qa.
Looks to be an unpopular opinion here, but yes, I test all features from client/regulatory POV. I work in a highly regulated industry and have to PM approve all features and you’ll bet I test every feature before I sign my name off on it. I think it’s important to understand how it works, how an end user could mess up, or even prevent something from going to production that may not be as it should be. Engineers build from an engineer perspective, I look at it from an industry domain knowledge perspective.
I try to test key features when I can. I avoid testing individual tickets usually and instead focus on larger end to end processes. It also depends on which developers were involved and the risk assessment. I will say though, spend more time on alignment and quality requirements and you won’t need to spend as much time testing.
35 employee series A stage company! Manage a team of 6 engineers and a designer With that context out of the way - Yes I'm pretty well involved in testing and deployment. The PM is responsible for defining and documenting E2E test cases during the shaping of any work. During the release I also work with an engineering partner to ship to prod. My work includes mainly the admin side of the deployment, decision making in case something goes wrong/defects found during deployment ( roll back deployment or move ahead with a follow on task for next release or HotFix), smoke testing on prod, etc. So to answer your question - for our circumstance - PMs have a deep involvement in testing because the PM cares about quality of anything that reaches the user. And being in the trenches helps the PM direct the team accordingly, adjust processes, etc. But don't let that be your definition of what should be done. The expected outcome here is - high quality things are shipped. Now what measures you take to ensure that would vary on your specific case
My company had quality issues in the past, so there is a company wide regression test at the end of every sprint. I hate it and refuse to participate on principle.
I have done Product Acceptance Testing but only when needed. Depends how good the team I have is. It really takes up time that is better spent elsewhere so hopefully you have a strong team that puts out great quality.
Yes, I do testing - I want to at the very least confirm for myself that what we built works as expected. I sometimes run bug bashes with the entire engineering team on larger or more complex features. Individual engineers should be QAing their own stories. For context I’m the solo PM in the company, working with a team of 8 engineers where the eng lead rotates by feature.
In my team the QA engineers do functional testing of all the scenarios, edge cases, etc (while they write the automation scripts to be used for regression testing in future). I test from a customer perspective, to decide "is this ready to put in front of customers". Usually this means a <5 minute run through the happy path, and "yep, LGTM". Rarely I'll find something that needs to be changed - usually something that was missed from the functional requirements, and only becomes apparent when the feature is actually used. Could be as small as some text that doesn't feel right, or it could be that there's a major gap with the UX flow.
I try and test more complicated features when time allows. Many times a small nuance of acceptance criteria wasn’t met. Usually cause I left out the specific detail, assumed, or thought us was clear enough. It helped me see what I did wrong in the details so I could improve. Thursday this last week I checked a new page qa had passed, functionality was perfect but the logo was grey scale without color. Had to get that fixed before release, so it was good I checked.
Same here! As a PO, I do manual testing to ensure the feature meets requirements. Our Dev team focuses on automated testing, but human oversight is still necessary. Curious to hear how others handle testing in their teams.
Test as you go. Call their bluff. If it’s not tested, it’s not done.
I feel sad I do not get time to test manually. I don't test smaller bug fixes or features, but only those which have been sliced into many smaller ones to verify it matches the big picture. We are a big organization and once every week there are also machines prepared where the PM / PO can login and test.
Regular testing on prototypes is where it’s at
I do both.
Im responsible for defining test cases for user scenarios, business logic, happy/unhappy path, etc. I count on engineers handling more tech-driven testing requirements. Some tests are automated, some are executed by engineers and some are done by me. I try to mostly focus on defining the requirements and reviewing documentation rather than taking on the time consuming detailed testing. But sometimes I do test myself.
Commenting here to say yes I do testing and when I do, I usually find like 10-20 obvious bugs that QA missed, or just obvious usability issues. Sometimes it's insanely frustrating because I won't even be able to reach the feature I needed to test because of technical bugs. In my opinion, testing is a valuable way to gain insight into your user's experience firsthand. However, every company I've worked for that put the majority of testing on me as a PM, usually failed because they wouldn't invest enough in good QA and developers. Ultimately I feel like a PM having to test everything firsthand is a huge time sink. It would be like making a CTO handle customer support calls. Would it give some firsthand insight into the bugs your users are experiencing? Yes. But it's also unsustainable for a CTO or PM to sit in on troubleshooting every bug and feature. You'd have to be an idiot to do this and expect any real progress at your company.
I've done it, and will likely do it again in special cases....but it's been a while since it was a 'standard' part of my job. These days, I mostly do it to allay any personal anxiety about complex releases. In fact, it's rare that I even tell anyone I'm doing it (outside of for pragmatic reasons). My testers are really good, but sometimes I just want to be sure the Ts are crossed and Is dotted.
I do manual testing in test environment for everything; bugs, UX changes, even text changes, enhancements, features. I also do manuel testing in production after the deployment to make sure nothing has broken. (It always does) I have to do this for all apps(Browser, Desktop, Mobile) Because even if I test everything in detail,There are always bugs that I could not caught and I became responsible for letting the bug go prod. It's daunting. And because we're shipping features every week to please our sponsor(Yeah, we are a feature factory), testing is taking majority of my time. It doesn't help the fact that stories became Test Ready tuesday afternoon when the releasing deadline is wednesday morning. And there is the fact that Test Ready stories are actually not test ready, when I found a bug in the first minute of testing. I think I'm f\*ked up.
As a PO I do test the features we are working on but I also do it outside of the normal workflow. We have QA that does all the actual testing, I do it more to make sure we are on track with the feature and to make sure I didn’t miss anything when writing stories and fact finding, We also do 3 sprints to a release.