How to build an effective UX Research Repository with Jared Forney from Okta

To answer this question we hosted a live session with Jared Forney, Senior UX Researcher at Okta.

Jared shared his journey implementing EnjoyHQ at Okta and his learnings when it comes to taxonomies, organizational change, data management, and ResearchOps.

Check the recording here 👇 Enjoy!

Transcription

[00:01:08] Jared Well maybe while we’re working through that, we can just kind of get a little bit of a high-level overview of kind of our roles

[00:01:14] Sofia So first of all, thank you, everybody, for joining us today. I’m very excited about this session and to have the opportunity to chat with Jared again about the work that he’s been doing. Okta. My name is Sofia Quintero and I’m the founder and CEO enjoy rescue. We are a research ops platform. We help designers, researchers, and product managers to centralize customer feedback and use that research data to streamline the research process and to share those insights with the entire organization. And Jared here, he’s a senior, a UX researcher, designer, and UX researcher at Okta. He will talk a little bit more about the organization and the context we’ve been working together for over a year, from the very moment when he decided, and the organization decided that they needed a better way to share research and to collaborate with different teams within the organization, all the way to the execution and the, and the success that he had implementing this. So in the meantime, I will let you direct to tell you a little bit about yourself.

[00:02:20] Jared Yeah. Awesome. So again, my name’s Jared Forney, let’s you mentioned I’m a senior UX researcher here at Okta. So Okta itself has an identity management and access platform. And so a big part of what Okta does is allow people allows anyone to use any technology and provide security and access controls around that technology. So a big part of what we do is dealing with a lot of different integrations and vendors and things and making sure all those things are connected together in a secure way. And as far as my role at Okta, I’ve been there a little more than three years. As Sophia mentioned, I started as a designer and I’ve increasingly now I’m full time research. And especially within the last year, I’ve also started taking on a lot more research ops responsibilities as well. And so a big part of that is one of our big goals in the last year has been to develop a centralized research repository for all of our insights. And I’d love to, and Sophia, this is an appropriate time. We start to kind of get into some of the details of what that journey has looked like.

[00:03:26] Sofia Yeah. So nobody, so I guess, shall we start talking about a little bit of the team and structure? So I know that Okta has been growing very, very fast and you have a growing design team and research team, so it will be good to know when do you decided to start it projects? They say, what was the team, you know, looking like at the time, has it or not all the structure and how you used to do research before that will be important to know as well.

[00:03:53] Jared Okay. Gotcha. And I have Sophia, I will say it looks like your, your screen and presentation thing might be in another tab. It looks like someone highlighted. So if you need to toggle your screen sharing settings off from earlier, but one thing I’ll, I’ll jump into here as well is kind of thinking about our initial need for a repository really stemmed on the universal question. Do we have any research on blank? Right. I feel like there’s probably a lot of people in this room. Who’s been asked that by designers, product managers, or other stakeholders, that’s been kind of a centralized question that we get a lot. And that was kind of the watershed moment where we realized we really need a way to centralize our research for us. It’s kind of primarily been about three things. It’s been about asset management. How do we manage all of the videos, the transcripts of recordings, the raw researcher, notes, diagrams, and images?

[00:04:51] Jared How do we centralize and organize all that information, knowledge management? How do we think about the output of all of that? The insights that are generated from all those artifacts of research and then also guidelines and templates. We’re actually a very small research team at Okta. Relatively speaking. The company itself is about 2,500 people distributed internationally. And our research team is four people. So we have some pretty, pretty big shoes to fill the product team. The design team that I sit as part of our research team sits with. And as part of product design, that alone has grown from about 10 people when I started three years ago to more than 40. So it’s about a 10 to one ratio for designers to researchers. Cause those are the people we most frequently partner with. So we have to do a lot with the team that we have and in that team itself, our research team is also growing, but maximizing our, our re our resources and our ability to do as much as we can with our head is very important. So yeah, I mean, those are some of the core reasons, you know, let us implementation path.

[00:06:00] Sofia Wonderful. So what we’ll do actually finally, I got a handle of this and essentially I’m going to share my screen so Jetta can help me to see, there we go. Excellent. So we have a little bit of context. I just wanted to tell you a little bit about the session. So, so we understand, you know, when we can ask questions and hopefully it will be an interactive session. So the ideas that we learned and it’ll be more about Jarrett journey, building a research repository and actually succeeding at it, we do the introductions, but essentially what would, what we’re going to do is to talk about our story, understand the challenges, go through the process and, and really focus on spending some time together for you to ask questions. We would love to be able to take this opportunity to share what we learned together, what y’all learn, and anything that is very specific. So if we go very, very nitty gritty, that will be excellent. Cause it’s all about learning from those lessons and building up, if you’re already in the process of implementing your research repository, or you’re thinking about it, hopefully, this will help you to have a bit about to your roadmap as to how to go about it. So I will stop this again very quickly and hopefully close the video.

[00:07:17] Sofia So Jared, could you talk about, and it would be the context of what you wanted to do at the very beginning. I remember our first conversation. You have this incredible vision that you not only wanted to centralize data and make it easier to collaborate on research projects and charity science, but you also wanted to involve all other team members in other teams in the organization, being able to tap into the sales kind of insights and then how the sales team was really interacting with customers and having a different perspective of customer support and marketing. It was a lot bigger than just research or user research. So can you tell us a little bit about why you have a vision and where you are right now?

[00:07:59] Jared Yeah, so like he said, there’s, you know, organizations, it’s not just product centric, right. Even though I sit on and work with our product teams, the ecosystem internally that supports Okta and helps it work day to day as much broader than just product. So I think a big part of what we’re trying to do as a resource organization and what the research repository serves as a centralized place for is a foundation setting a foundation for one, helping us get our own house in order and organize the information that we already have, but then much more broadly as a long term vision, helping us more effectively collaborate with customer support with our customer success managers, with our developer experience teams, all of these people who also reach out and touch our customers and work with our customers on a daily basis that we may not necessarily have as easy visibility into day today.

[00:08:54] Jared So a big part of what our research repository is helping us do is to make those connections. And when I, when I say connections, I can kind of mean that in a lot of different ways, there are some of the more nuts and bolts connections like integrations. You know, we were working on integrating with a number of different tools that we use internally. And then there’s also connections that it’s just helping us bridge that gap, you know, with our own, with face to face contact, right, helping us meet with our marketing teams and our customer success team saying, Hey, we have this centralized tool that we’re gathering all our product feedback about. We’d love for you to take a look and see what we’ve learned that can help you have better communication with your customers today. In addition to that, if you ever get an email from a customer that has feedback or is experiencing a pain point and you don’t know what to do with it, now there’s a place where not only they can get direct access to our team and the people who can help make that change, but it also helps give a lot of agency to that success manager, sewing, Hey, I’m helping be a more effective ear to the customer by, you know, brokering this relationship.

[00:10:03] Jared So that’s what a repository has really helped us do, you know, and, and making those connections.

[00:10:08] Sofia Yeah. Was it difficult in any way to, for example, get access to other tools to integrate, for example, so some teams are in different companies, different cultures, there will be more, you know, wary of, you know, giving access to other people to data. How was in your case where the sales team happy to say, yeah. Connect Salesforce to your tools so you can get, get access to all my data and so on. Can you tell us a little bit about that?

[00:10:34] Jared Yeah. So I’ve learned a lot about procurement in the last year. One of the things that taking on a little bit more ops responsibilities day to day, and it has been a big challenge and it’s, you know, there’s definitely some growing pains associated with that. Not only in, in getting a centralized repository tool on board, the practical things, right? Like securing budget, getting the net, we’re a very security conscious company, obviously, especially us, you know, all, all companies have security and security controls in mind, but given it’s a very, it’s a foundation of what we do. It’s, you know, very central to any new tool that we bring on board. So I’ve learned a lot about the importance of identifying partners, not only with the tools that you’re, you’re bringing into your organization and partners like yourself, that I’ve worked with closely over the last year, but also really learning a lot about the importance of partners within your organization.

[00:11:29] Jared Getting to know the members of procurement and legal and finance and security teams, better learning how I can be a better, more communicative partner in establishing. Here’s why we need this tool. Here’s everything I know about it. What questions do you have for me in terms of being more communicative in that sense? So that in it while, while those integrations, you know, do take time, especially if it’s a big, far reaching tool like Salesforce, I think what I found is what I’m learning about centralizing as tools, not only helping us with the repository, but it’s helping us for any tool that we want to procure and use in the future, because we’re always looking at it through that lens of all of the company and how we can be a supportive partner and how this tool can benefit everybody, not just the research team.

[00:12:17] Sofia That’s super interesting. And as you were creating those relationships and in going through the procurement process and learning, is there anything that you can share with us in terms of, you know, I wish I knew this, you know, one of those lessons, like a wife, I had to do all over again, the whole procurement process and trying to implement a research repository, what would be some of the things that, that you will do differently or perhaps that you would advise people to do?

[00:12:42] Jared There’s also been a lot, I’ve definitely learned a lot from this process. I think the biggest thing, and I’m sure it’s a thing that a lot of people are thinking about, or maybe worried about when they’re thinking about launching a repository is taxonomy, right? Like how do I categorize all this information? How do I group it? What if I make the wrong tags? What if I get off the ground with the wrong set and things are just as messy as they were before. I spent a lot of time upfront thinking about taxonomy, researching different ways I could seed it. And just really a lot of pre-work went into that. And I think what I realized after a lot of effort was that I needed to be willing to accept the idea that like, it’s not going to be perfect out of the gate. It is very much an iterative process and where, what ultimately was most successful for us was looking back at our past studies and using those as guidance for how we seeded our cortex on these.

[00:13:43] Jared So a lot of what we did, we have tons of legacy data. We are, I mean, we’ve been doing research for probably, you know, most of the company’s 10 year history. So we have a lot of backlog that we’re bringing into our tool now, but a big part of it was looking back and seeing what, you know, maybe it’s affinity, diagramming exercises that we did, right. We have reams and reams of classifications and tags that we’ve made of that data, looking for patterns across studies. What are the kind of core things that we’re talking about? Okay, we’re talking about features of our product a lot. We’re talking about particular types of users. A lot. We use common terms there. I’m using that as a foundation for a taxonomy rather than sweating every little detail from, or looking for external sources and inspiration to bring in.

[00:14:31] Jared But mostly just understanding that taxonomies are iterative process. It’s something that you’re constantly cleaning and refining and not to worry too much about it upfront. It’s also a really good opportunity to learn about your organization internally, like thinking about the, for us in our field. I am sure this is true of a lot of folks. We have a lot jargon, right? We have a lot of acronyms. We have just like a dictionary. We actually have an internal dictionary full of Africans. And that’s something that like, when I look at it from a taxonomic standpoint, I’m like, how can I not only tag the isn’t make them reference well, but also make it more approachable for someone who’s new to the company. You know what I mean? Rather than it turning into an alphabet soup, how can I use this taxonomy to bring clarity to my organization?

[00:15:17] Jared Things like that, other things that I’ve thought of that I wish I would have done maybe to have a little bit more internal training and governance at the outset. And I use governance more, somewhat narrowly in this context, I use governance in terms of who has access controls into what degree. And when, you know, when we’re trying to kind of bring a certain vitality to the repository, to get people excited about it, get people aware that this tool exists and that this is here to help benefit them. We want to give people access particularly to our core insights and summaries of data. Some of the challenge has been a lot, particularly a lot of our stakeholders are like, I really would love to see the conversation. Like I love to read transcripts and I’d love to get closer to the customer, but we also recognize that we have to be careful about how much data we expose initially to people without proper guidance and understanding about how to interpret those results, you know, in an unbiased way.

[00:16:23] Jared So a big part of what I did after we integrated our repositories, I did an entirely self sourced course on enjoy HQ. I went through the entire product end to end and did like an hour tutorial video broken up by chapters and publish that internally in our internal Wiki. So any new product manager or designer or anyone else in a product team who wants to get more granular access to the data outside of our summaries, we give everyone read only access by default, but if they want to do more hands on closer work with the data, I recommend they go through that course, or I’m also in the process of doing live teachings too, like one-on-ones or group sessions where people can learn about the tool and ask about best practices and that sort of thing. So, you know, if I had looked back on it now, I probably would have had more of that ready from go to help generate excitement, but also to kind of make it a little bit easier to gain access and give people the right tools from, from blog.

[00:17:25] Sofia I super unsure. There are a lot of other lessons will come in as we talk, but I was wondering if you could, one of the things that I really liked about your approach and how you went about these was this idea of like gradually stages faces, you know, starting, like we’re going to start small, we’re going to prove that this actually works and then we’re going to start adding other people. So you have all these very clear plan of how you wanted implement a repository within Octa and ensure that, you know, everybody has a plan until they get punched in the face. So I would love to hear, you know, can you describe a little bit about that initial plan and then how things have changed? What have you learned from those interactions in that kind of facing approach?

[00:18:09] Jared Yeah, I mean, I think phases like with anything, you know, it’s the, you know, they say about best laid plans. So it’s, it’s one of those things where a lot of the work that I’ve been doing with respect to the repository and thinking about our research operation broadly is from a career perspective is very much pegged to my internal goals and metrics and those that the team. So kind of hitching our wagon, you know, connecting that to our core team, success has been important for two reasons. One, it gives us milestones to look at and to it also underscores how important an initiative this is for our entire team. A lot of the challenges with that as things change and scale changes, and especially given the current global situation, it’s changed a lot of our understanding and in some cases, prioritization of the things we choose to work on.

[00:19:05] Jared I think one of the biggest challenges that we’ve really quickly understood the vast scale of it is just how, how long it will take to get all of our backlog research into the tool. It’s something that it, isn’t a simple copy and paste operation, right? It’s one of those things where it is a lot of it involves us. Re-examining our own research data, our own insights, how, because we, that didn’t necessarily have a taxonomy attached to every insight where every call for every customer. Those are sorts of the sorts of things where we kind of almost have to not completely redo our research, but we have to re-examine it in a very critical way. And that’s been a challenge because again, there’s only four of us total and we have other studies and responsibilities in addition to bringing this archive in. But we also recognize the value of doing that because it helps us understand what work is still relevant today, what work, you know, one of the things that people are coming back to even two years later, it’s so foundational that people are still looking at this research versus, okay, maybe this usability test we did was, was relevant in the moment.

[00:20:16] Jared And it helps somebody turn a corner, but like six months to a year from now, we’ve moved so far beyond the original scope of that test. Maybe we don’t need to keep that, you know, so there’s definitely opportunities for introspection there. I think that’s been kind of the most jarring thing is like, Oh, getting, getting the, the full body of work even from just the last year is a pretty big lift.

[00:20:39] Sofia Yeah. Do you feel that in order to be successful with the repository, you’d really have to bring all that data together or do you that it’s more important to get into the new process of how people collect, share, and work with that data, even if it’s just the most recent, like how do you, how do you see that?

[00:20:58] Jared Yeah. That’s also something that’s really important to emphasize. Like, you know, it can be a specificity in task to attempt to bring in your entire company’s history or history with respect to research into a tool like this. And I think there’s also some arguments to be made that that may not necessarily be valuable, right. For some of the reasons we talked about like research deprecates right. It goes stale. It doesn’t become relevant anymore. And I think that’s very much an important part. We did have, like one of the early decisions we made was like, we needed to make a very clean break after a year. Like we are going to be very selective about anything older than 2019. There are, there are still a handful of studies that may be relevant, but we’re not going to bring everything in. It’s just, it’s too much effort for like the payoff wasn’t there.

[00:21:47] Jared So the, the big initiatives that we did were that like, let’s start with the last year and see where we are after that point, see how long that takes. But the other significant change was that for us, as soon as the tool was live, everything from the future forward goes into the tool. We’ve made a very conscious effort that like getting it centralized and being really stringent about that is, is already paying dividends. I just had a study. I finished just yesterday where I was working on a particular aspect of our mobile experience. And I had some customers that I was talking to and I happened to do a search. You know, it’s still early days yet in terms of work that we have in there. But my, one of my colleagues did a completely unrelated study where they mentioned just offhand some things about our mobile experience. And I was able to immediately incorporate that and stuff from April, from our company wide conference, we had a lot of people requesting like mobile support for our platform. And like I was able to fold that stuff in immediately. So even insights that are only a month or two old are immediately relevant to us. And so we’re already seeing kind of a snowball effect of the power that even research in the recent member in recent memory can really, really help our organization a lot

[00:23:06] Sofia That is rainy. And so I would love to start the Q and a session I would love to have, because we have a lot of questions. We have almost 300 people here. I’m excited about this, but before we move to the QA, just one last question, what caring, you know, what are they key, tangible benefits that you perceive so far throughout the journey? It is important to know that it’s a process that takes time, of course. And so is it worth it? A lot of people think about it while the research that when you do my grade and this and this and that, you know, it’s a word fit. So can you tell, like what have made a difference?

[00:23:40] Jared Yeah, I will say it’s absolutely worth it in whatever, whatever format, whatever approach you take, whatever tool you use, the exercise, just the mental exercise and the process that you go through and thinking about the centralization. It is time well spent. I can definitely vouch for that. Some of the things that building a centralized research repository has done for us, one of the biggest ones is it’s pulled data out of silos, right? We all have research in Google docs and in box and Dropbox and slide decks and all these scattered places. And those little 15 to 20 minute sessions of digging around, looking for insights for somebody across all those platforms adds up in a big way. So pulling them out of silos and being able to point to a centralized point in a repository or even making it self service is already just hours a week saved.

[00:24:35] Jared And that alone it’s reduced the need that kind of as a followup to it’s reduced the need for unique research. So we’re still doing a lot of studies. It really hasn’t like necessarily at this point, we’re still developing and getting backlog in, but it is reducing the amount of studies that we’re doing over and over because we can’t find or effectively utilize the data we’ve already done. And if we are doing studies along similar lines, it’s with a deeper understanding of the problem. And it’s allowing us to be much more focused on our approach. It’s shortened discovery periods. So a lot of by centralizing those tools and data, it’s allowed us to really start to think about research, not as a series of tasks, but more as a workflow. So from recruitment to the actual scheduling of participants, to running the sessions, capturing the notes, organizing the data, analyzing the data, delivering the insights, it’s all happening within the context of a single pipeline rather than scattered through a bunch of tools.

[00:25:34] Jared So that ended itself to saving us time because it’s all stored in one place. It also allows us to produce evergreen insight, right? We have themes that come up over and over again in terms of benefit and themes that emerge over and over again. And before we knew these themes were happening as a team, but we had trouble communicating them because we had to grab them from so many different places and show three or four studies and say, see, these are all related. Now we can actually correlate kind of all the core insights from each of them studies and gathered them in one centralized place and show them. And I think lastly, the last major thing that it’s done is it’s really helping us to be more proactive. I think our research role historically has been generally one of more of a reactive stance. So somebody says, okay, I’m thinking about developing a new product or feature, and I need to go to research to understand the problem.

[00:26:30] Jared And that’s fine, you know, and we still, we still take a lot of requests of people who need research for things, and that’s great, but I think ultimately what this tool will help us do is help us identify trends before people come to us. And we say, Hey, we are noticing this pattern of behavior across all of our customers or across the segment or across the studies we’re doing. We’re seeing this theme and we believe, and here’s the evidence that this is a real problem. And this is something that I think we should stall solve from our approach from a product perspective. And I think that’s really, really powerful of giving research a voice in a, in a really effective and organized way.

[00:27:09] Sofia That’s excellent. Thank you so much for sharing that. I would love to start with questions. We have a lot of them. So the first time we have, I think it’s Sandrine. I think it was Mary as well. They ask what is the actual process for entering past research and new research into the platform. And we talked about it before, but perhaps you can give an example or for example, this research project that you just finished, right? Like what were the data that you were adding to the, to, to the repository or the data that came out of it? And also, do you spend sometime during the week to bring previous research, or you did all of that at the beginning when you started the process?

[00:27:48] Jared Yeah. That’s and the, this is obviously going to vary platform by platform. So, I mean, I’ll speak specific to it, some specifics to enjoy HQ, but in general, what I try to do from a repository perspective to make my life a little bit easier is I try to always start entering data in the tool itself. So like the most common method that we use is, is customer interviews. A lot of customer interviews, you know, they’re, they’re generally always remote given our kind of international distribution, but a lot of times I’ll take my core notes and enter them into the call itself. We usually conduct them over zoom. And so once I’ve taken that customer call, I always make our recordings of our calls as well. And whether or not that includes audio or video, screen-sharing like if it’s a usability session or something like that varies.

[00:28:37] Jared But the core thing is I always start with my raw notes. I go straight into the tool. So that way they don’t just, you know, they don’t get stuck in a document somewhere. So those will go in and then I’ll also take the recordings and upload them directly into the tool. As soon as the call is finished, you know, zoom usually sends an email receipt saying, Hey, your recording is ready. So I’ll bring that into the tool as well. And one of the things I really enjoy particularly about using a Q is the ability for it to generate transcripts automatically. So we used to do a lot of transcripts. We used to request them separately because a lot that saves us a lot of time in terms of being able to go back and reference a particular session. But getting those instantly allows me to actually start in the, marking up the context of our conversation while it’s still fresh.

[00:29:26] Jared So I usually try to do that within a day or so of the actual call itself, so that I’m always keeping a running record of what’s happening, what themes might be emerging, that sort of thing. And I’m also maybe when I’m working within enjoy HQ, I’m also using the analysis feature in particular, which is kind of like a combine almost Trello style board that keeps track of all of the highlights that I make and puts them in a centralized place. So a lot of times, by the time, if I have 10 calls, I am, I am by call four or five, I’m starting to become aware of, of themes, right? So I might even start roughly sketching those themes out in the analysis board. And as I’m making highlights, I’m kind of thinking, Oh, that kind of falls into this loose bucket. And that way I’m kind of already building a very loose taxonomy of the work I’m doing so that when I’m done with all my calls, when I go into analysis, I’m not doing a ton of raw tagging by hand I’m instead thinking in these big semantic thematic blocks. And so the combination of those like automated AI based transcriptions and enjoy HQ gives, and the analysis board really helps streamline that workflow a lot for us.

[00:30:44] Sofia Oh, that’s super big though. Thank you so much for sharing that. So we have another interesting question here. So three key one. So how are you measuring success? Any KPIs, like how do you know this is working? And a lot of people ask this question and the benefits come in very different shapes or forms. So I would love to hear like how you’re thinking about this.

[00:31:04] Jared Yeah, this is, this is a perennial, this is a perennial challenge, right? Like how do we take, particularly in our, our speaking to my research operations specifically, we’re a very qualitative house and that’s something that we are constantly evolving and we’re starting to think a lot more quantitatively and enjoy Esq. And in part is helping us take that first step towards quantifying more of our qualitative research through this taxonomy, right? Being able to generate reports and dashboards based on this feedback allows us to tell a really powerful story over time, particularly with our, our mobile feedback, you know, being able to scrape from the various app Google app and an Apple play stores. You know, those sorts of things allows us to tell that story over time and what the biggest issues are. I think, where going forward, what our organization is going to be looking at a lot more is our, our, how we can marry our product metric feedback with enjoy HQ with the qualitative, how we can put the two together.

[00:32:07] Jared We’re collecting quantitative data like our data coat, your data team, and the product team as a whole is starting to collect more and more quantitative data on like, who’s clicking on. What, where are they clicking? Where are they dropping off that sort of thing, which is great about telling you the what, right. But it doesn’t necessarily speak to the why. And that’s kind of where I see ourselves right now, where we come in is helping address the why, and really starting that kind of symbiotic relationship, that kind of cyclical nature of what and why, and marrying those two kind of, and approaches together into one kind of unified collaboration.

[00:32:44] Sofia Got it. So, and I also had other research teams and there is another layer as well. Like they, they want to know, you know, whether I know the people around in different departments or teams are actually interacting with the repository, whether or no they are reading the research summaries stories, reports. So as a, as a broader, we’re thinking about this all the time and how we can provide like pretty great reporting on the engagement that they holders have with enjoy HQ as a platform. So who is reading the research with asking questions? What are the insights that get the most attention? All of that stuff is just a really a key focus for us when it comes to engagement within the organization, which is just one dimension of why this might be successful, but absolutely unable to drive, be able to understand what is the impact of having all this knowledge available in the decision making. And the quality of those decisions is, is a tricky thing to measure, but extremely important. So I have another one here. This will be perfect. Actually it says, how will you manage security levels? You know, we do a lot of people research that you’ll not be viewed by just anyone. So of course this is different for every company, but how you think about that in terms of access and so on.

[00:33:59] Jared Yeah, this is, this is something that’s really, really important and speaks a little bit to the governance stuff I mentioned earlier, you know, the, obviously there’s with the increasing prevalence of, you know, GDPR and a lot of other privacy centric regulations, it’s exceedingly important for us to be able to protect our customer’s data. And it is a foundational part of Octa as a company. I think that from a research perspective, when it comes down to is we’re still very much at the point where we are very hands on with the people we work with in terms of maintaining access to the repository access controls are something that’s absolutely vital. And as I mentioned that hands on training is something that I’m doing a lot more of to make people aware of not only how to use the tool, but also how to safeguard the information.

[00:34:46] Jared We do. Other things within enjoy HQ itself. We anonymize a lot of our data in the summary format. So the actual people who are generating the quotes, we don’t specifically attribute them to the specific individual, unless that person has the necessary access controls to view the underlying source material. And that’s just important from a standpoint of like, we do want to make it available if someone needs that context, but we also want to make sure that if they would like to engage with a customer based on their feedback, that they come through us, because we ultimately are the brokers of that particular relationship. And we want to ensure that above all our customers feedback is respected, but their privacy is also respected. And their ability to give candid feedback is, is intra. And it’s just like inexorably linked to the, the security and the safety. They feel to be able to give that us feedback. So a big part of it is a combination of physical access controls within a product and ethos, and ultimately like education, you know, both from my, from a research team perspective. And also our company as a whole has a number of different trainings and perspectives on data privacy and things that, that are org wide.

[00:36:00] Sofia Wonderful. I’m going to follow up with this question that is a little bit related. So it’s more about how do you promote or engage people that are actually doing research, but they are not researchers or part of the research team. So let’s say marketing or somebody or product manager, they just want to do research on their own. Like how would they participate with the repository?

[00:36:21] Jared Hmm. So this is something also, that’s a little bit of an evolving process for us. I think that our closest research allies are the people who are doing research their own research right now are our designers, right? I think designers are probably the closest to us from a, a product standpoint and a researcher standpoint. So we often collaborate with designers and helping them design their own studies. Most of them are usability focused at the moment just because it’s, it’s the most common thing they particularly need help with are at a crossroads and a design decision. And they need some input, but also it’s very pragmatic. It allows them to kind of move forward and help unblock them on certain things. But while we’re doing that, we’re also really encouraging them to think critically about the problems that they’re approaching. We’re, we’re teaching them methods for writing research plans, how to ask unbiased questions, all these like research fundamentals, right?

00:37:15] Jared And at the same time, we’re also working with our product management partners because they’re talking to customers already. So a lot of what we’re doing is saying, Hey, we’re giving you access to this repository, but we also kind of have a shared community aspect to it too. It’s like, we’re opening this data to you because we want to make it accessible to everybody, but we’d love if you help us give us more input. So, you know, if you’re talking to customers, you know, if you can invite us to those conversations, if you’re not already, or also, you know, if you’re capturing recordings of those conversations with their consent, help us add to that library. Or if you’re getting emails from customers with feedback for those into the tools so that everybody can get that same level of visibility across the board, because every new piece from every participant in every party that’s added to the tool and categorize benefits everyone, rather than it just benefiting that one individual.

[00:38:10] Sofia That’s excellent. Thank you. So how about thought of questions here, but I’ll see if we have enough time to cover more. So can you tell us a little bit about your reveler stack? So essentially the question is, you know, how you combine in your, his queue or repository with things like neural or mural, or you said testing tools, like how does that work?

[00:38:31] Jared Yeah. So this is also, you know, and an ongoing, as I’m taking on more and more ops work, it’s increasingly what’s changed about my thinking with respect to our, our research stack is it is always from the perspective of how does this integrate with enjoy HQ? You know, do you have API hooks available into your product that we can use to integrate into this to create smoother pipelines? Does the data exportability and data portability is a huge part of, of any tool that I review? How easy is it for me to get data into and out of this tool, should the worst happen if the tool goes under, if there’s a problem and I need to be able to jettison all of my data and get it captured and into another tool, how easy is that speaking specifically to our stack? It kind of breaks down, I would say, into, into three discrete parts.

[00:39:28] Jared So one part of that is our recruiting pipeline. So I guess I can, I can mention names here, but we, in the past, we’ve worked with responded to IO. We’ve worked with user interviews as well as a combination of internal tools that we’re using an internal platforms that we have for recruitment. And a lot of that, of like which we use and in what combinations depends on the audience that we want to talk to, how much time we have, how much budget we have for the project. There’s several levers that affect the upward bound recruiting pipeline, things that we’re also looking on the upper part of the funnel with respect to recruitment, our consent centralized consent management, as well as overarching governance controls and things like that. The second stage is kind of the middle ground that actually like running the sessions themselves, that can also vary depending on time tooling, budget methodology, but in the past, we’ve obviously used video call Valls video calls like zoom and other platforms for face to face or interview style sections.

[00:40:36] Jared We use Miro internally for workshopping, but we’re also exploring something I’m really excited about potentially exploring collaborative or partner design with our customers. So being able to take assets and being able to work together in a collaborative design collaboratively together in a space with the customer to help better capture their thinking, visually other things that we’ve used, we’ve used Validately for moderated unmoderated, user testing amongst others. So that’s, there’s a whole bunch of other capture tools that we’re using in that middle stage. And then the late stage is more about the organization and the repository capture insight distribution. So that’s where enjoy cue sits today. It sits in the centralized part of being able to store all the results of that research mural also plays a role in this because we often run collaborative remote workshops with our stakeholders to help make sense of the data and organize it, and also to give people a closer handle and really lend to that tangibility of the insights rather than just being handed a report.

[00:41:45] Jared A lot of what we’re really trying to focus on is how can we make our stakeholders not only an equal partner in the analysis, but also owners of the results. So those are kind of the three broad buckets that our tooling generally falls into. And I’m, I’m a tools guy. I do try to be careful that I, that I not don’t you, you don’t make it all about the tools, but I also just love exploring new platforms. And I’m always really excited to see when there are new developments in a space where it can really help open some doors and opportunities for us. And I think automation is like the last stage of that too, that we’re, that we’re starting to, we’re doing a lot of workflow based things, exploring some of those options as well.

[00:42:25] Sofia Right. And yeah, from, from our side, I integrations is our thing. We understand how important it is to have an easy data flow between all the tools, just to make that work a lot easier to manage. And for those that are interested, we are planning this year, integrations we use or soon will usertesting.com. I was going to say that to Jared before, but I, we are building soon. So you can connect your assume account and bring all that. So yeah, we’re trying to build as many integrations as possible that help you just forget about the amount of work and really concentrate on the analysis and the sharing of the insights. And so on. I have a lot of questions here on textile. Let me send, I have to have Jared for another, ask me anything session, like just Jared talking about it. Cause we have a lot of questions, but there’s one here that I would like to, to, to take, which is concrete steps for a solo researcher that wants to do this. Cause it’s dated, right? If you are, even if you’re in a smaller organization, like if I’m the only researcher, how do I go about this? And I feel that you have both the experience of implementing something in a large organization, but at the same time we were the first one and the only one for a while doing this. So maybe like,

[00:43:42] Jared Yeah, it’s, it’s scary. It’s intimidating. Right? Like it’s how do you take, how do you begin to categorize all this data? Especially if you have a lot, some of the things that I, like I mentioned a little bit earlier, I think one, like don’t, don’t sweat it. Like I know it’s, it’s scary to think about, and, and we’re worried about making mistakes and about creating a lot of potential noise amongst your data, but you know, any taxonomy or any organization and attempt that you make is going to be an improvement over the current state of the world, which is to say there’s no there’s there’s little organization. And a lot of siloing. The second thing that I would say is create a seed, but don’t, but don’t overthink it when, like I did what a lot of what we started. I sat with my team and if you’re working as an, this is something that, you know, you’ll have to do on your own.

[00:44:34] Jared But the advantage that you might have as an individual is you have a, you know, 20,000 feet view of everything that you’ve done. So, you know, your data best. And so going back and looking through whether you’ve done affinity, diagramming sessions, or you’re looking at keywords that came up a lot in your summaries or in your raw notes with, with your participants or common pain points that came up in usability testing, start with those. And if, even if that is a little tough, I didn’t even just start. I actually just honestly started with the information architecture of our product. Like I a is a great starting point because right, we all have it. Like we all have our products have navigation of some sort. And as soon as you start thinking about taxonomy at that high level, you’ve already got a framework and that skeleton in place where you can say, okay, like underneath security, we’ve got API access management, we’ve got mobile administration.

[00:45:32] Jared We’ve got, and then we’ve got directories over here and inside directories, we’ve got applications and people and integrations and all those things, the gears will start turning and you’re like, okay, I know we have these potential issues that we’ve spotted in this part of the product before. And you’ll feel it start to tumble out. So a combination of using your product as a foundation, using your path research as a foundation and not sweating it too much, the last part of it, I will say is kind of commit to making it an ongoing cadence of scrubbing. So like I would, you know, I try to do once a month to go in there and take a look at the tags and the properties that we’ve assigned and see, how are we doing? Like, are there a lot of orphan tags and properties that maybe don’t have any assignment to them that were just made?

[00:46:23] Jared Or maybe there’s only one or two, are there ones that are combined like for us, it’s admin and admins with an S so like, can we consolidate those into a single tag or property and look for patterns that way? Cause one that’ll help keep things tidy. And two, it will also make you more aware of how your team talks about your own product and that’ll also help build your taxonomy over time. So those would be kind of the, the tips I would give. I know that’s a lot, it’s something that definitely is not a big bang. Like you do it gradually

[00:46:56] Sofia Slowly. Correct. So I know we’re coming to the end of the session. There’s so many questions that we couldn’t, you know, take today. And I will, I will. I copied all of them in a document. I promised you that we either want to do an interview with Jared or another session with him because he has so many amazing things to share with us. I wanted to definitely, I’ll definitely get you in another session. Everybody’s like going crazy with questions and it’s amazing. So I wanted to share, just share my screen again, just for one second and then Jared, again, you can help me out. Oh no, I’ve gone. Look at this. Look at the lack of security we’re back. So I just wanted to share something before we go, Oh no, this is not what I want to share present. Excellent. So here’s my email address, what I would like to do with everybody, because we had a lot more people than we thought.

[00:47:52] Sofia And, and I know there’s a lot of questions is that you can email me and then we can do a couple of things. We can have a session together. When we talk about the challenges that you have today, or perhaps sharing some of the things that we have learned as well from other teams. And in Jerry’s experience, you can send me an email. If you want to also join another session with Jared, or do you want to join a session? We doing these sessions with different customers as well in other webinars that are upcoming. So if you want to come into another one, just email me and say, add me to the list and the same for a demo. If you want to see what we can do and essentially show you the tool around and how to use it, I’m happy to do that. And I’m sharing the email and not necessarily a link because I want to do this personally. I want to be able to get to know you, you took the time to be here for an hour or so, and you got a lot of questions. So I want to make sure that those questions get answered. So I just wanted to share that. And of course, say to Jared, thank you so, so much, because means full. And I believe that, you know, we will have to spend seven hours here to go through everything.

[00:49:01] Jared Yeah. I’ll, I’ll add, I’ll add to, you know, I’m available on Twitter at Jared Forney. If anyone wants to tweet at me or message me on LinkedIn with any questions, I’ll do my best to kind of answer them as I go. I, I definitely will want to take a look at the questions here. I’m planning to do a couple of series of articles, two in the near future about, about the things I’ve learned and, and more about kind of the context of things that we’ve talked about today. But I want to take the time to, to really think Sophia she’s been a really great partner for us. So I’m always happy to do these sorts of activities because I think it’s such a powerful platform and a powerful idea that I think a lot of people will resonate with. And I’d really encourage people at any level. And at any budget, you can do this. Like you can build a repository with any kind of mindset you just have to start. And that’s the hardest part, but I’m happy if you want to reach out to me like there’s my Twitter handle there. If you want to reach out to me with any questions, I’ll do my best to try to get back to you as soon as I can, with more assistance on that.

Originally published at https://blog.getenjoyhq.com on June 16, 2020.