Subject as Sales
“I just don’t have the time to consistently go in and find Bowery subjects to use as comps. If that data was easily accessible, I would use Bowery subjects exclusively in every report.”
Discover
In order for an appraiser to justify their valuation of a subject property, they must support their conclusion by using examples of data from other similar properties in the same area. These snippets of data from similar properties are known as “comparables” or “comps” for short. Comparable data can take several forms, such as building expenses and net profit, taxes, rental rates, and more. Comps are necessary in any appraisal, and they can often be difficult to find and verify.
The Problem
The Opportunity
By far the most time-consuming comparables to find and input into an appraisal report are sale records and details, known as sales comps, because of how fragmented and unreliable the data in the public record is. This means that in every report, an appraiser will spend anywhere from 10 to 15 minutes per comparable searching for and validating the data within. A typical set of sales comps in a report is made of around 4-6 different sales records.
At Bowery we have appraised thousands of subject properties across the country. Roughly 40% of the properties we have appraised also happened to have sold within the past three years, which would make them eligible to be used as sales comps in other reports. Because these sale records come from properties we have appraised, this means the data is easily verifiable and extensively researched as a symptom of our appraisal process. Compared to public sale data on external sources, the proprietary sale data we have on our subject properties is air-tight. In other words:
“Brokers, owners, and agents do lie.”
“There’s nothing more trustworthy than comps that come directly from a report we’ve done.”
The Hypothesis
We believe that appraisers will have to spend less time researching a sale if they know it comes from a Bowery-appraised Subject. In turn, this feature will reduce the amount of time appraisers spend searching for and selecting sale comps, and increase efficiency and revenue per appraiser.
The Technicalities
The goal: Reduce time spent on-app searching for and verifying sales comps, increase appraiser efficiency and revenue.
Timeline: 2 weeks
Tools used: Paper & pencil, Google Forms, Miro, Figma
Process: The good ol’ Double Diamond
Understanding the Problem
I began by conducting five user interviews with appraisers of varying levels and expertise across asset types and geographies. I asked questions like:
What makes a good sales comp?
How do you search for and select sales comps to use in your reports?
How trustworthy are sale records on a public website like CoStar or Zillow?
I also ran a survey of 8 questions sent out to the entirety of the appraisal team at Bowery. I received 18 responses.
“We know so much about the subject properties; detail-wise, it is automatically better than most of the comparables we research from a third-party lead database like CoStar. If the property is encumbered by leases, we have that information too, which is extremely valuable. The knowledge of the subject details provide more credibility to it as a comparable as opposed to a “standard” comparable. In short, we have all the data!!”
123 Address St. was appraised by Bowery. It was marked as being done "on app" in Salesforce and the property is now listed as Under Contract. On a scale of 1-5, how trustworthy is the property data stored on Bowery's app for 123 Address St. to be used as a comp?
On a scale of 1-5, how likely would you be to use the data contained on-app for 123 Address St. as a Sales Comp for another report?
“I would use Bowery data over anything else.”
Define
Who is Affected by the Problem?
After assembling all of my research, I plugged it into Miro and sorted my data into key takeaways, pain points, and key quotes that really helped me illustrate the heart of the problem (and I have scattered throughout this case study).
From there, I wrote out a few insight statements (see right) I had crafted using evidence from the research phase. These statements helped me to summarize the data I had on appraisers in a way that allowed me to paint a stronger picture of the user who I was ultimately trying to help.
Verification of comp data with a secondary source is necessary because of the lack of trust appraisers have in brokers and listing agents
Appraisers trust Bowery data more than any external source, even CoStar or Crexi
Appraisers are confident using Bowery app data, even if they are aware that the app and the final PDF are not always 1:1
The Persona
At Bowery, we have existing personas we use to help define the broad needs of specific subsets of appraisers. These were created during a research initiative in 2019, and were updated earlier this year. The persona who seemed to be the most affected by the problem we were trying to solve was Casey, our Homegrown Appraiser. Casey was trained at Bowery on our proprietary software application, meaning she is an on-app power user. Homegrown Appraisers typically take on the biggest majority of on-app work (meaning building types and geographies supported by our technology), and would be the ones most benefited by the ability to reuse their on-app appraised subjects as comps in other reports.
User Stories
From my synthesis, I was able to draft three user-centered stories to help anchor my ideation to specific, well-scoped problems to solve. Using Casey as my starting point, I wrote the following user story:
As an appraiser working on Mixed Use & Multifamily properties on-app, I want to quickly and easily access Bowery-appraised subject data while on the application, so that I can use it as expense comps for my current report.
Task Flows
I also assembled a Task Flow diagram for the existing expense comp search and selection process to illustrate how time-consuming and bloated the current flow for appraisers is. This helped me identify bottlenecks in the process that could serve as opportunities for my eventual designs to improve appraiser efficiency. The flows were also important to help contextualize and illustrate the problem space for other stakeholders during the ideation phase.
Develop
Crazy Eights
Before jumping into any sketching or wireframing, I gathered a small team of stakeholders together and ran an ideation exercise in Miro. By getting other designers, as well as members of product and engineering into a room early in the ideation process, we were able to generate a wider variety of ideas on how to solve the problem and I was able to create buy-in for the solution I settled on. In addition, this would prevent any surprises for the engineering team since I had people in the room who could provide insight into the feasibility and technical challenges of different solutions we proposed.
Wireframing & Prototypes
After settling on a solution from ideation with my teammates, and some additional brainstorming with pen and paper to iron out the details, I moved from low-fidelity into high fidelity wireframes in Figma. After a few days of pixel-pushing, I had prototypes that were ready to test.
Usability Evaluation
I assembled a research plan in Notion, summarizing a lot of the above details. The solution I designed would help on-app appraisers like Casey quickly and efficiently identify Bowery appraised subject properties that contain suitable information to be used as expense comparables. Appraisers would also be able to easily validate the details of the on-app subject information to ensure accuracy. The hypothesis that we were testing is the same as mentioned above:
We believe that appraisers will have to spend less time researching a sale if they know it comes from a Bowery-appraised Subject. In turn, this feature will reduce the amount of time appraisers spend searching for and selecting sale comps, and increase efficiency and revenue per appraiser.
I drafted a testing scenario that put the user in the middle of an on-app appraisal report, where they had just begun the process of searching for sales comparables to use. I found five appraisers who resembled our persona Casey and ran them through remote usability evaluations via Zoom, and the designs scored an average of 4.8 usability points out of a possible 5. Appraisers were overall very receptive to the concept of reusing subject data as expense comparables, and all five users were able to identify Bowery-appraised subjects in the database using the icon we use in maps to denote the subject.
Deliver
The Hi-Fi Solution
How did we do?
After user testing, a senior researcher and I used the task flows I had assembled in order to run a GOMS analysis on the effectiveness of my designs. This exercise would help us evaluate whether we had hit the goals of the project, which were to reduce time spent on-app searching for and verifying sales comps, and to increase appraiser efficiency and revenue. After crunching the numbers, we found that the Subject as Sales Comps feature:
Made appraisers 71% more efficient
Would save an estimated total of 343 days, 2 hours, and 33 minutes across all appraisers at the company over 3 years
Would allow a potential 92 additional reports in one year to be completed across all appraisers at the company
276 additional reports in three
Increased estimated yearly revenue per appraiser by $4,000
$368,000 yearly across all appraisers
$1,104,000 across all appraisers after 3 years
Lessons Learned
Seemingly small features can have massive impact. Running a GOMS analysis with the help of my senior researcher really helped quantify the impact that this tiny little blue star could have on the day-to-day workflow of appraisers.
During wireframing, I ran into a lot of issues building out some of the tables you see in the wireframes above. It got me thinking that we may need a better table component to be designing tables in our application, as they’ve always been a bit of a sore spot. So I built one myself! Case study coming soon.
Next Steps
One important thing to note is that this feature does not support our appraisers who work off-app, in geographies where we have very little on-app data. I’d love to spend some time talking with these off-app appraisers to understand how we might begin to pull them off Excel and onto the application, and how we might incentivize them to start adding data from their geographies into the data flywheel.