Professional focus group facilitators call it various terms like group think. When I was tending bar in NY putting myself through school I listened to folks in the bar pass around ideas or observations that spontaneously became facts.
How are you evaluating your UX or features? How big or diverse is you sample? Is your success with you MVP based on two very loyal early adopter friendlies?
There is a very trusted UX application vendor who I like but often tests in coffee shops based in a very technical town with an average age of 28. Not very diverse and a sample that thinks they are smarter than whomever developed the app. The result is predictable, ever first evaluation shows significant would needs to be done and guess who helps them out.
This is not planned fraud but also not a faithful review. We all have done it, just last year I had a board application I did used by three boards that already were using the governance school that I based it on. My MVP test was great but the market for the application was limited to Boards that already were faithful to a school of governance, the mass market potential was very limited.
Coffee shop or bar talk is limiting, like minded talk like minded. Courage is to stretch outside and broaden the scope of your sample. Broaden it in ways that define new personas both by demographics and technical perspective. Technically there are users of a competitive product, non-users you need to convert and technical phobic or friendly minds.
This is stuff that makes sense and we all trip up on. Testing needs to be robust, samples need to me deep and diverse. Testing both market and pressure is always the first thing deleted from the project plan when time lines compress. Just don’t do it!