Chapter 128 - THE PUBLIC COMMENT
At the public hearing, a vendor displayed a chart showing a ninety-eight percent satisfaction rate for its emergency-information app. The numbers were real, according to the survey it cited. The chart did not mention that responses came from people who had successfully downloaded and used the app. Residents who couldn't install it, access a smartphone, or understand its language settings had never been asked.
Alma raised her hand. "Ninety-eight percent of whom?"
The vendor described the sampling method. She listened, then asked how many residents had been excluded before the first question. He didn't have that number. The facilitator requested a corrected presentation showing the population surveyed and the limits of any conclusions.
Nia offered the center's own attendance data, with similar cautions. We knew how many visitors had requested phone assistance and how many appointments were completed. We did not know how many people never arrived because they had been discouraged by the instructions outside the building.
Afterward, the vendor approached me and said the hearing had made his company sound dishonest. I told him I wasn't qualified to judge his motives. His chart had answered a narrower question than the title suggested, and the revision should make that clear. He looked unhappy but said he'd submit the additional information.
The hearing room was full of people carrying charts. Some wanted a larger budget for emergency services; others wanted assurance that no new contract would waste public money. The vendor's presentation looked professional, with blue bars and a nearly perfect satisfaction figure printed in the corner. I watched the audience nod before Alma asked her question. The number had the beauty of certainty, and certainty was welcome after months of confusing information.
The city analyst, Grace Wu, had also reviewed the survey. She explained that people who couldn't use the app had not been included in the satisfaction question. The survey still said something useful about people who successfully used the service. It did not establish what the broader community needed. She offered to help the working group design additional measures using phone and in-person options without requiring residents to disclose more personal information than necessary.
A resident accused the vendor of trying to deceive everyone. The facilitator reminded the room that the method and presentation could be criticized without inventing motives unsupported by the available evidence. The vendor agreed to revise the title of the chart. He didn't enjoy the correction, but he returned for a second session with more detailed sampling information.
At the second session, he admitted that the company had been rewarded for growth in active app users, not for reaching people who couldn't use the technology. That incentive had shaped what the team counted. Simon asked whether the city might require contractors to report both successful uses and barriers to access. Grace said such a condition could be specified in future contracts and reviewed for feasibility. The public office agreed to examine it.
Denise spoke near the end. She'd tried the app once during a break at work, found the login confusing, and gone back to the phone line. Nobody had asked whether she was satisfied because she'd stopped before the survey appeared.
"You can put me in a chart if you want," she said, "but don't put me under people who didn't care enough to finish. I was at work."
The room grew quiet. The vendor thanked her for the information and asked if she would describe the difficult step in a user-testing session. Denise said she might, if the session was scheduled after her shift and her time was compensated. The city recorded the request rather than expecting her to do professional testing as a favor.
Back at Open Door, Elena used the hearing as part of volunteer training. She showed the group two statements: "Most app users who completed our survey were satisfied" and "Nearly everyone in the neighborhood is satisfied with emergency information." The first was supported by the cited data. The second wasn't. Trainees were asked to explain the difference in language a visitor could understand.
Tessa, now helping with youth outreach under supervision, said the first sentence still needed the response rate because people who hated the app might be less likely to answer. Elena agreed. They added the rate, the method, and the relevant limitations to the training guide.
That evening Luca found me reading the revised report. He said a board he'd once served on had treated market research the same way: a favorable number became a story about everybody, even when the sample was narrow. He regretted how often he'd accepted the easy version because it made a decision feel cleaner.
"Do you wish you'd questioned every chart?" I asked.
"I wish I'd known which question to ask before making a promise based on one."
Lily interrupted us with her bridge graph. She had drawn error bars after a science teacher introduced the concept. She couldn't fully explain the mathematics yet, but she knew that repeated tests didn't give exactly the same number. When Luca asked if she was proud of the new model, she said yes, then added that she wanted to see what happened in more humid conditions.
I looked at her careful uncertainty and thought of the hearing. A number could help us learn without being forced to provide a verdict. The people missing from a chart might be the very ones a public program most needed to hear.
Grace's team proposed an additional survey that could be completed by phone, on paper, or in person. It would ask residents how they obtained information during a recent advisory, including those who did not use the app. The draft avoided collecting names or addresses unless a participant separately requested follow-up. Denise reviewed the questions after her shift and pointed out that one answer choice assumed everyone had chosen not to use the app. Some had tried and failed. The team added that option.
Alma asked whether the survey would be offered at accessible transit locations as well as libraries. The analyst said the sampling plan was still under review and agreed to bring the concern to the research group. A better survey would not eliminate every bias, but it could make the limitations clearer and reduce preventable exclusions.
When the first summary appeared, the results differed from the vendor's original chart. People who used the app successfully still reported high satisfaction, but many residents relied on phone or in-person assistance, and some described access barriers. The city used those findings to strengthen its multi-channel requirements rather than declaring the app worthless. Different services met different needs.
At Open Door, Tessa asked whether it was possible to collect too much feedback. Nia said yes, especially if organizations kept asking people to repeat problems without acting on what had already been learned. Each new survey needed a defined purpose, a responsible owner, and a plan for telling participants what happened. The group agreed to include that requirement in its future recommendations.
At home Lily showed me a revised bridge chart. She had included the failed test instead of deleting it because Jonah said the failure explained why they'd changed the brace. I thought of the wider survey and smiled. A flattering number might be comforting. A complete picture was more likely to help somebody build something better.
A city council member later cited the revised survey during a budget meeting. She did not claim the new data represented every resident perfectly. She described the methods, the groups reached, and the limitations that remained. The committee approved a review of phone access and in-person staffing rather than transferring the whole budget to one technology. Afterward Grace thanked the people who had challenged the earlier chart. Denise said she'd be happier when the service actually improved, and Grace agreed that the next report needed to show actions, not only findings.
I saved the summary beside the older version rather than replacing it in our archive. Anyone following the decision could see how a question about a flattering percentage led to a broader picture of what residents needed. The change was not a repudiation of every app user who'd been satisfied. It was an acknowledgment of all the people who had not appeared in the first picture at all.
Tessa added a note to the training exercise asking volunteers to identify the population behind every percentage they quoted. It became one of the simplest and most useful questions in the guide.
At dinner, Lily presented a graph of her bridge tests. The newest model had carried more weight than the older one, but she'd also tested it under different humidity conditions. Jonah wanted to call it the best design yet.
"We only tested it three times," Lily said. "We can say it did better in those tests."
Mom smiled into her tea. Luca asked whether the class would appreciate that qualification. Lily shrugged. "They might find it boring. But it's the right label."
May you like
The revised vendor chart appeared a week later. The satisfaction figure remained, now accompanied by the response rate and the statement that people without app access weren't represented. The city added telephone and in-person options to its planning requirements. A flashy percentage had become one fact among several, which was exactly where it belonged.
END OF CHAPTER 128.
