Speaker 1 0:00 Good afternoon, everyone. Welcome to How to your startups measure up. This is presented by Autumn. My name is Holly Lundgren. autumns online professional development manager. And I'll be your staff host for today. All lines have been muted to ensure high quality audio and this session is being recorded. If you have a question, you may enter it at any time using the q&a feature. We also hope to make today's session as interactive as possible. If you have a comment or questions you'd like to discuss out loud, please click on raise hand and we will call on you as we're able and we'll go ahead and unmute you to make it interactive. Want to take just a moment to acknowledge and thank autumns 2020. Online Professional Development sponsors, we appreciate the ongoing support. At this time, I'm going to turn it over to our distinguished panel. Our moderator today will be Robin razor. So I will turn it over to Robin. Thank you Welcome, guys. Speaker 2 0:59 Well, thank you, we as you can tell, we look a lot more distinguished haha, it pride pre COVID. But what we're going to try and do is is do everything we can to make this as interactive as possible. Everybody that's that's, that's listening in, hopefully has already seen, you know, our BIOS, if you're not familiar to who we are, so I'm not going to go into that. And hopefully, you've already seen the slides, although I will go through them quickly. And then as as we mentioned, we're going to we have a q&a. And I'm going to try to watch all these q&a questions and have a period of time where people can ask questions. And at some point, we're also going to try to see if we can do the raise hand things and enable people to actually, you know, speak we have a lot of people on this call. So we'll see how this goes. I also believe that the autumn impact metrics group in my life known as the survey and metrics committee is listening in. And hopefully they're taking notes as well, because of course, we're talking so much about metrics. And so we're all hoping that we will achieve some results here. So let's see if I can make this thing move. Okay, here we go. Unknown Speaker 2:22 Wow. Speaker 2 2:24 Okay. So hopefully, many of you have seen there was an article published in Nature Biotechnology, in February 2020. And actually was talking about the biotech living and, and the Walking Dead. And I suspect that I'm not the only one that received multiple copies from other faculty, my bosses, other people. And, in particular, what got a lot of people's attention, was the language about that universities, were creating maybe too many startups, did these startup zombies actually really contribute to economic development? Or, you know, were they creating products, perhaps that the universities were not doing a very good job in creating high value companies? So then this became a question and we were planning on doing this, this workshop at the autumn meeting last year, because all of us on this panel in various, for various reasons, have this as a very important issue for us. And the question really is, you know, are we the profession contributing to the problem? You know, are we being consistent in how we define our startups? Are we recognizing that all of us have, you know, are at different universities with different different pressures? How are we measuring success in our, our startups? And when I realized today, when I was looking at this, what are successful metrics? What this really meant is how are we defining success for our for our startups? In addition to how are we defining success in our tech transfer offices, for the venture development that we're doing? And also it does beg the question, when all these metrics started years ago, we were mostly talking about patents, and the kinds of technologies that would go into startups that were patented technology. And now as we all know, we have software, we have data, we have content kind of startups, and they tend to be least I believe, they tend to be a bit different. Many of us, including me, had been involved in the startup and our excuse me, the survey the autumn survey and metrics for many years, almost since the beginning. and have spent a lot of time working on the definitions. Because the objective was not only to collect the data, but also to make sure that we were when we were collecting the data that we had apples, we were comparing apples with apples and not apples and oranges. And I think a number of us are concerned that we're not always using the autumn definition, not only for startups, but we could get into a discussion at another time about other metrics such as agreements and other things. So I've underlined some of the things that sometimes I think, become questionable when when universities are reporting their startups. And so for example, you know, when I talk to my counterparts in other universities, it's interesting to me, even though there is that written definition, different people have interpreted it different ways. You know, is it a startup when it's has an option? What if it's a foundation? What if it's a nonprofit? What if it's a student startup, all these kinds of things that perhaps are open to interpretation, and also could contribute to kind of the question about how we're measuring how we're reporting our metrics. So what we wanted to do today, and I will, we're gonna go in the order of I'm going to be done here in a minute, we're gonna go in the order of Jack, John, and then Oren, each of them are going to talk about five minutes, and then we really are trying to open this up to q&a and discussion. Is, is does the autumn definition really provide us an accurate representative representation of how we define startups? You know, do we need to change that definition? You know, how are we do we need a different standard for how we define startups? What metrics about our startup? Should we measure? How do we measure success not only in our startups, but in terms of the kinds of programs that we provide for creating and helping our startups? You know, it begs the questions. Do we need one metric? or, excuse me, one definition for all of us, recognizing that some of us are private schools, some of us are public schools, some of us have, you know, more of an economic bent and others, maybe one you know, it, maybe one doesn't work? And then are we putting too much emphasis in terms of startups? And lastly, we came up with this as we started talk, should there be a nine points to consider document that the university has put together about responsible metrics, that would include startups as well. So those are sort of the things that I'm hoping you all are thinking about. And our panelists will kind of present their thoughts initially, and then we'll go into the q&a. So Jack, you're up first. Unknown Speaker 8:11 So Robin told me I have five minutes to talk about everything I know about startups. And then she told me, that was way too much time for me to talk about everything I know about startups. But you know, I think I think really, there is a, there is some pressure for tech transfer offices to start up companies. And so you know, whether it is a technology that is very promising technology, and it's not advancing commercially, and it needs to move. Or if it's, you know, some other reason to start up a company, the tech transfer office is, you know, under pressure to move that along. On top of that, I think government agencies put fuel on the fire by providing an instrument through SBIR or STTR, is to advance technologies. And so faculty come into the tea to and say, hey, you know, I'd like to start up this company to access some of that capital and public universities, our hands are tied, when you start up a company, you know, no longer can you use public entity to advance a private company. And so that it's a real challenge. And so is a company A company at that point in time? I don't know. I went from, you know, the University of Michigan, one of the most prolific startup company, universities there are to Cleveland Clinic where we've got one of the most active corporate venture capital teams and it was really interesting. You know what, what I learned, we built a returns driven team at Cleveland Clinic. The three guys that I hired, one had 20 years of venture capital experience. As one had nine years venture capital experience and the other had 10 years experience at strategic. And, you know, we talked a lot about the ad companies in our portfolio and what we would do to advance those companies. And after a few months, I found that I was really sideways with some of the guys on the team. And what I found was that reputation really, they weren't not interested in talking about some of the companies in our portfolio as a venture ready company. And so, you know, is as we, as we built out the promotional material, you know, we had to figure out, what do we talk about? How do we talk about these companies? And the reality is, there's two phases of these companies. There's the innovation side, and there's the returns on investment side. And I think that's where we get a little bit tied up. And I think the article, the biotech article is, it has some really interesting points in it. But I don't agree with everything that is stated in there. And I think that it mixes up a lot of different parts, like Robin was talking about, you know, is a digital company, should it be put into the same discussion? And I think that the metrics that that are presented by Autumn, you know, when is a startup, a startup? I think, I think they're okay. I think really what it is, is, what's the purpose of the startup? And that's what I always come back to is, you know, what's the purpose of the startup? Is it to advance the technology? Is it to create jobs? Is it to take advantage of funding that's not available to any other resource that it is only available to a company? So? So that's my perspective, I think that you know, it's a real challenge for tech transfer offices, I think that it should be more focused on what's the purpose of the company, as opposed to Is it a company? Because you do have a good definition? But what's the purpose? What's the use of the funds after that point? That's it, Robin. That's what I have. I'm good. Speaker 2 12:15 All right. So, John, you are up. Unknown Speaker 12:19 Alright, so just to get it all on the table. I'm a pro startup. I think that startups are important. I think one of the reasons for our being is to help those startups get off the ground, and it is probably the most expeditious productive way to advance some of our technologies. And given that 80% of what I do is in the life sciences, it is absolutely the route by which many of our companies have to choose to go otherwise, it just isn't going to be advanced. But I think Key Item definition is insufficient. It's I mean, if all you're trying to do is say, when did it incorporate? And did it incorporate around technology from the university? It's okay for a definition. But at all, we've already experienced a number of universities abusing that by simply counting how many incorporation documents They've filed around option agreements that faculty have asked for. So I think we have to have a much more rigorous definition. And I understand that, that puts me in a very, very small minority. But I worry about our ability to actually demonstrate impact. If, and I know people are gonna say, well, some software companies don't need much more than that. That's true. And I don't do many software companies. So okay, you got me. But the vast majority of the technology that we develop at universities needs to have a team of people who can actually reduce it to a proof of concept or proof of principle so that it can actually be developed into a product that an investor could actually examine, invest in, and ultimately take the market. And that's not normally going to happen within a university. Maybe Publix can do that. I know privates can't, I mean, we're just, that's it's verboten. So my point is not whether or not startups are good or bad. They're good. I love them. It's just that what I don't like is a definition that isn't very rigorous. And I think that we ought to be looking at sustainable startups companies that we could, if up a newspaper reporter or a politician or one of our senior administration at university, wanted to actually go visit their our doors to knock on with human beings that are not employed by the University, working on advancing the technology, and it actually ends up advancing the local economy at the same time. And I think those are all positives. And I would like to see our definitions and the way we look at it be a bit more rigorous in defining what a sustainable venture looks like. That's it. Speaker 2 15:15 Okay, before I go to Oren, I just want to make sure that if you're going to ask a question, please use the q&a, not the chat. Because I'm having it's enough for me to try to watch both. I'd rather just watch one. So Oren, you're up? Yeah. Hey, everyone. Unknown Speaker 15:33 Yeah, these are all really good points. And actually, I don't, John, I'm not even sure if you're in the minority on this. I think a lot of us have some of these same, same questions. It's just like, from my perspective, it's just a really complicated question. On one hand, it's clearly good, that we have a uniform definition. And, and that that uniform definition is used in sort of honest and straightforward and consistent ways. Like with, you know, consistent metrics, there's always that phrase, you get what you measure, I think consistent metrics serve a variety of purposes that are actually really important to all of us. Probably most importantly, they allow for honest self assessment, and also external assessment from others by comparing your peers which can not only serve to show whether you need to be doing a better job, but also can help identify other universities who have best practices that you might want to try and emulate. And Lord knows, like when I joined this industry 15 years ago, that's the first thing I did was take the audit data, and slice and dice it to figure out well, how are people doing what they're doing? Like? How are they handling that money, and mentioned that that few staff, or generating that many licenses of the startups with that few inventions like what's going on, so you can need that. And also, the data set, it's been really interesting to see the audit data get used by third parties over the years to find really interesting analyses that I think can help grow our profession, I think, John, to your point. There's certainly a demonstration of impact as a profession, which these metrics are really important for as well. So we need to demonstrate that by goal was a good idea, we need to demonstrate that we are being good stewards of the innovations coming out of federal funding. So boy, Robin, if I cede my time to John solders from a raised his hand, do I lose it from my five minutes, Madam Chairwoman, or do I, Speaker 2 17:21 he doesn't get to raise his hand until you have three more minutes? I was Unknown Speaker 17:26 really hoping you would say that. Alright, anyway, but on the other hand, I do and John and I have disagreed on this. I do think that different universities have different pressures and objectives. And some of that's based on what their public or private, or whether they're in a one of the sort of big startup states or not, whether the middle of the country, or the coasts. So you know, for instance, at Columbia, while we do use a very consistent and rigorous autumn definition, and we're consistent year to year, I would describe our goals at Columbia, as encouraging the spirit of commercialization entrepreneurship, letting people learn how to do this, even if that first time they launched, it turns out to be the wrong idea, the wrong problem, or the right idea on our own problem, helping to recruit and retain faculty and students who are interested in entrepreneurship, being attractive to industry and VCs generating bragging rights for New York City, New York State. And actually, the weird thing about our region is I think, New York City in New York State care actually care more about volume of startups than they necessarily do over having one or two big successes. So we're sort of aligned in this, like, let 1000 Flowers bloom mentality. And it's been less as an office about revenue generation for the university or being part of the big winners. And so, you know, and also different universities have different operating strategies. And so So for instance, you know, at Columbia, we actually prefer to do an option with a term sheet, as opposed to a license for as long as we can. Because it's much we find it much more transactionally streamlined, it gets done faster. And also the companies honestly pit it a lot of the time in the first six to 12 months anyway. And so this way, you can both kind of agree on what the business is, as opposed to having to write a license that encompasses all possible patents. We're also happy to, to license if there's only one party interested in the technology, we'll let them have it. Like if it's a bidding war, we'll be selective but frankly, if no one else wants it at a startup comes along and wants to license it. We are happy to let almost anyone Garona we obviously do the normal checks to make sure it's ethical, but we'll let an untried entrepreneur go run with it. As long as we have reasonable diligence in there that is near term and enforceable and we actually do terminate the startup deals all the time for missing diligence milestones in the early years, but also, at least in our experience, we found you never know like we've we've had Walking Dead companies that we've used the term Walking Dead For, not that we they were fake, we just never thought they'd go anywhere only to later have them pull along and get, you know, 20 $30 million of VC investment, years after we thought they would be dead, or even get sold or IPO. So, you know, on one hand when when when universities are faking the data, when they're like, creating their own startups, not really with any intention to grow them, taking an option, there's no employees, and they're just sort of duking the stats. That's not great. And we should expose that and we shouldn't stand behind our peers. But I'm wary of imposing a tougher national standard. I actually think the standard we have as long as we're all enforcing and consistently is good. The question to me, it's really interesting here is what else should we measure? Because John, I think that that's where we get to your points of, there's all sorts of things like it's a yes. And version, it's, let's count startups in a consistent way. And we're doing that already. But what else might we want to know? So that so that we can do those same kinds of longitudinal and cross geographic comparison analyses, on more things than just how many? But how big did they get? How many employees did they have? How many of them raise any professional funding? How many rounds of the raise? How much did they raise? Like, did they start in the same place? Did they stay in the same place? That would be fascinating for us to know, as an industry? And so I'm fully supportive of that. Not with a national standard, tougher baseline attached? Speaker 2 21:32 So John, did you have a comment? Before I get into some of the q&a? Unknown Speaker 21:39 I just the question I had was, or the issue I have is, to what point when we start just counting things where people raise their hand, they may have taken an option and, and called it a startup? What's the point? What's if those things are are born dead, or walking dead, and I understand I absolutely agree, some of these things take a long time to launch before they're ready to go. But why count them at that point? I mean, what's what's the real advantage to that other than having a big number? Because if somebody were to say, Hey, I would like to go visit, you said you did 25 companies last year, I'd like to go visit 20 of those. And you go, Oh, wow. So 20 of them. Maybe a little hard, but here's five that actually raise money that you can go talk to, and people are gonna look at you and go, What did you do 25? Or did you do five? Which which is and that's that's where I get crosswise with this? And, and that's my point. Speaker 2 22:51 Well, let's, let's see how we go. I mean, I let's we do have a couple of questions. But this does beg and I'm channeling my inner Lita and Kathy Kuh, who used to refer to the metrics as as an arms race. And I think this is what John's alluding to is an orange. I'm I do not believe people are using the autumn definition when they're counting startups. I think startups are being counted in an apples and oranges way at universities, because I I hear from other people as to how they count startups. And it's not how I would count to start a startup. So there's, you know, there's an example right there. So couple the question, you know, and I'm going to bring up a comment or question, actually, from the impact metrics group. First, before I get to some of the q&a, is, I think, going to John's question about, are there things that we can measure in the startups and maybe this goes to, if they hit a certain point, then they get counted? For example? Do should universities report on the progression of commercialization? You know, clinical trials, prototype funding? I know, in my institution, we do try to track their funding, whether it's seed funding venture, Angel, even SBIR STTR, we do try to track that as one measure of potential success. Another measurement that people well used to do is what's your percentage of companies that are still alive after five years, 10 years, you know, things like that. Do the panelists want to comment? Are there potential for things that we say okay, until the company reaches x, it's not counted, or is that a feasible way to Unknown Speaker 24:50 think how do we do that? Do we raise our hand or just start talking? Just go for it. I wanted to start talking before anybody else did. You know this kind of gets back to you? My point earlier, it depends on what the purpose of that startup was. And you know, what metric is important. And, you know, not to, you know, flip flop on the whole. I mean, I agree, John, you know that, you know, the metric is kind of wonky. When you're talking to an investor, right? When you're trying to make an investment, and you say, you know, how many startups, they think a company is a company with a CEO, and an honest to goodness business plan, etc. That's the way they look at it. So the internal metric that maybe Tito's use is okay. But it can't be the same measurement that you use externally. And so, you know, I think, if you're saying, you know, what is the long term sustainability? Did they get an A round of funding? That depends on if they were a venture quality company? Or were they just an extension to the lab, and they needed an SBIR to solve for x and then licensed the technology off to somebody else? It's the purpose of the company, which is going to drive the use of funds, in my opinion. Unknown Speaker 26:04 But why would a company that was flipping the technology through an SBIR? Why would you count that as a startup? What was the point? Unknown Speaker 26:13 I definitely definition it's a Unknown Speaker 26:14 startup. But all it was going to do was license the technology. It wasn't I mean, Unknown Speaker 26:20 and, and so then, I guess that's my point, right? Is that if you if you talk purposefully about what's the design of the company, what's the purpose of its, its being incorporated, then you're going to be more likely to be tracking the rate, the rate metrics? I wouldn't call it a company, I'd call it a product development startup or something else. Right. Well, Speaker 2 26:42 that. So this goes to, could we end up with categories of companies? And let me bring in one kind of question comment. I wanted to answer the question. Alright, go ahead and answer, then I'll bring up this. Unknown Speaker 26:59 You, I do think you, and this is where Warren and I do agree on is I think you want to you want to measure lots of things. For us, you know, obviously, I want to, I want to measure how much money was raised. I also want to track the product development. So you know, is it in preclinical development? Is it gone into human clinical trials? You know, it hasn't raised a management team. I mean, we look at all those things. But the point where we counted as a company, is when it's been financed by third party. And third parties don't include the federal government with STTR as an SBIR, so we just don't count those. But, you know, after I get off this call, I'm going to be on a call with a group that has an outside management team has taken a license to technology from Yale, but has not raised money is nowhere near final product status, where they're actually going to have something there, they need to raise money. I'm not counting that yet. When they raise money, I'll I'll count it. But until then, they're they're hoping they're going to be a company. And the whole purpose of the call that's coming up at five o'clock, is to talk about how do we go about raising money because without it, we're dead. Unknown Speaker 28:19 Right? So I guess, let me just take that example. And I, John, I totally get what you're coming, where you're coming from like that makes, that makes a lot of sense. Depending on what one is trying to measure and why. But if I put if I put myself in, in, in the shoes of Columbia, and the shoes of the university administration, and the researchers if we have the vast majority of our faculty or students who would go out and launch a company that they then raise SBIR funding for, and then spend six to 18 months of their lives spending money that SBIR funding, when they tell their friends and their family what they're doing. They are by and large, can be saying I started a company, I'm an entrepreneur, their business card is gonna say I'm the CEO, XYZ. Now, when the Deans and the department chairs of the university are out talking to their donors, and talking to potential new faculty, and bringing people to meet their entrepreneurship advisory board. They're going to be bringing these people out. And those folks are going to be saying, I launched a start up on attempting to enter this space. And I've done my product market fit and going through Ico or what have you. But I'm not there yet. I haven't raised my money. And the university loves to bring those people out to meet the trustees and to meet the alumni and to come speak at events. There were a newspaper article about them. And so from our perspective, I don't see it as my job to ever say to those people, you're not running a startup, you're not an entrepreneur. I may not internally put them in the same bucket as the thing that has is like when I when we think about our companies that are the most promising the ones that are raised money from 5am, or arch or Lux or pure tech or others, certainly are, you know, more exciting at the moment. But to me, they're no less of a startup. Unknown Speaker 30:14 And and we would certainly bring those people out, we certainly would encourage him, we actually have classes and programs just like you do to create those. It's just I'm not reporting that the autumn or as a county for autumn, but internally, hell yes, we want to have those kinds of entrepreneurial activities going on, we're going to try them out in front of the faculty in front of the administration, the deans and whoever, and people love to see that stuff. And that's great. And I want to do that. But, you know, I'm not, I'm not going to put a number out there, that includes that simply because I'm afraid somebody's actually going to challenge me. Where are those companies? Speaker 2 30:52 So I think one of the things that's come up and, and I apologize, some of you, I know where you're from, and other people I don't so Doug Hawkshead, from Arizona, just mentioned, are we getting? are we confusing, formation of companies meaning number of companies with success of companies? So for example, you know, I'd argue, okay, if you have an IPO or an acquisition, and that company never makes your product, but you made a bunch of money. Is that a success? I don't know, I think we would consider that differently. And then Luis Martinez, sorry, I'm not, I don't know where you're from, mentioned this issue of when we have companies that get started. And really the initial management, I think this is what John was talking about, is is the graduate students or the postdocs. And we well, I suspect many of us have seen those kinds of companies where they, they they come out from the faculty members lab, but the faculty member doesn't leave. You know, I have a faculty member at my institution who actually likes doing that, because he's taking the point of view that he's training the next generation of entrepreneurs, you know, we're in the triangle, we're not in Boston. So this is a way to further develop, you know, early stage entrepreneurs. And if they're, you know, they get the company to a certain point, well, then, hopefully, you know, more, how do I say, seasoned? management comes on? Does that count as a company as well? It's almost sounds a little bit about what what, John, you're talking about, you know, your your meeting this afternoon? So is there, do we? It does sound like we're conflating numbers with success. Anybody want to make a comment? Well, I Unknown Speaker 32:47 think it is, at least with some of the federal grants. It starts with them. Right? And, you know, thank goodness, there is federal money, but at the same time, they're, you know, their currency and Capitol Hill, is companies created and jobs created. And so I think that we do I think that we say, you know, we ring the bell 10 times for for a new startup, and, you know, with no real perspective as to who's managing it and where they're going to be in two years. Unknown Speaker 33:25 So, Doug, no, I don't, first of all, just because they raise money doesn't mean that they're going to be ultimately successful. had too many failures that way. Yeah, there's an opportune, there's a higher probability that they ultimately become successful. But what I'm, I guess what I'm saying is that it, it really is not a company until it has independent management, it has capital, that it can actually execute our program, and is executing against a plan that involves my technology. And I think the capital is important. Now the capital can come from product sales, okay? Robin has made that point over and over and beat me over the head with it is that you can have a software company and it can be out there selling, it doesn't have to raise venture capital, you can sell and raise and sell finance, I would count that. I mean, I don't have a problem with counting something like that. I don't do many of those. So I don't have enough experience with it. But I just think that that is that a venture isn't a venture until it has independent capital, a management team that it's not university faculty, and it has a license to our technology and a plan that execute against it that's I just think you have to have those three things and and that's where I differ with the autumn definitely. Unknown Speaker 34:40 So I guess it is I think this is sort of a couple of different questions along these lines in the q&a and I think the sort of blends a lot of them but I'm not sure about confusing, successful the startup, but more that like it seems like as an industry, we seem to have a tendency towards a very reductionist view on what these terms mean? Like that there has to be, you know, a startup means what a single thing. And I can imagine that there's an autumn definition for what that startup is. And that's a really important for like how to become a startup, maybe we should be having different terms for what it is we're trying to measure. So I can imagine, like a launch startup versus a funded startup versus a commercialized startup, for instance. But like we do this, you know, it does universities, we count things have different success all the time, like, a faculty member is a faculty member of the university. But some faculty have a half a grad student. And some faculty have labs with 40 people in them. We have we win the universities count how many students are there? Someone made, the point does, you know is not all students graduate. And that's true. And that's a different statistic that is trapped, though. So you wouldn't say we're only going to retroactively deflate the actual number of students in the university by what percentage of those will graduate? Like, it seems to me keep track Unknown Speaker 36:14 of people who actually graduate within a certain period of time. Exactly. And Unknown Speaker 36:20 so maybe that's the right model, though, is we Isn't it like even on this call, we're using a single terms, startup to mean a lot of different things. And maybe it's time for us to grow up as an industry and start saying, there's no, there's nothing like when a when a postdoc when the world's most naive and inexperienced postdoc in a lab, decides to start a company that has almost no chance of success, both because of their own lack experience. And also, because the technology they're trying to employ is far too early, and fails to raise money on it for the first six months. That doesn't mean it's not a startup, it just may not be a startup that we're going to send the trustees to. But it's still he's still that person is still an entrepreneur, he or she is still an entrepreneur, and is still trying and, and so we can use, maybe if we back away from what is the startup mean. But rather, what kind of a startup is it? And and maybe that would be better. Maybe that would be more productive conversation as an industry as opposed to sort of hammering away on this one term. Speaker 2 37:27 I think there's a couple and I think everybody can see the q&a. So not just me, but a couple of people have made some really interesting comments about should we categorize companies, for example, you know, they're out there, the investment, their management, their technology development? Are there other metrics that we could use, either to categorize the companies, but also to follow success for the for the companies? And I, I wonder if that's really where we're going? Well, maybe, let me just ask, point out one other comment that I think is really important, and I'm going to lose it. There's a lot of stuff coming in right now, that the other thing that I think many of us are very conscious of is that we either have separate groups within the tech transfer office or another group within the university, that spending a lot of time working on helping these fledgling companies. And there is some need to demonstrate where that's going, how successful that is, and how people are doing. And the question is, we need to track it for that too. Because otherwise, your management is going to say, Well, wait a minute, why are we spending money on you know, your new ventures group? Or why do you have these mIRC? Or why are you paying for ers? You know, and perhaps what's happening is we're getting hung up on just saying, Oh, well, we did X number of companies, but we're not following through in terms of the success. So I think there's this combination of demonstrating that all this effort we're putting into is turning into something, but then how do we demonstrate it? Unknown Speaker 39:21 Gee, I've shut everybody up. Unknown Speaker 39:25 Well, alright, so I'll wait into this one. That's exactly I mean, it's, it's, it's precisely why I tend to track how much money has been raised, how many rounds, how many employees, where they're located, etc, etc. because that way I can say that they're real, and that there's if somebody wants to go see something, that there's actually a there there. I, I, I would not be opposed to having, you know, sort of some sort of definition that is or or Data gathering that is not tied to a unilateral definition of startup. So you could have different stages, for example, or something like that. The problem I have right now is that people look at the number and they go, gee, John, you only started 10 companies last year, and do did 25. What's wrong with you? And I'm like, Yeah, okay, do counts a lot different than we do. You know, and I have to explain that every year. And the provost always asks, so if you were to count just those things for which a company was formed and took an option or a license? How many is that? The answer is probably closer to 25. I said, but I can't send you anywhere to show you those. And he goes, Oh, okay. So 10 is a good number. I mean, I have that I've had that conversation with every provost. And I've done six Provost now. And so it's just a conversation we have, and they understand that my number is going to be low. Because that's all I'm counting. But probably on average, we're, you know, net net worth probably on a par with, you know, if we normalized by research dollars, we're probably on a par with those who are like us. Speaker 2 41:21 I mean, this does beg the arms race issue. I mean, I have more of an arms race issue more recently with how we're reporting agreements. But that's a topic for another time. Another comment or question was, and again, this becomes something for the impact metrics group, I think, is maybe I'll just ask this panel, if you are going to do your annual benchmarking for your management. And I think all of us do in certain ways. What are the three to five metrics that you use when you do your benchmarking? And I will answer it for my institution. It used to be agreements, startups, patents, and invention disclosures and revenue. I don't do agreements anymore, because I, I don't do agreements anymore. I will do exclusive licenses, if I can get that kind of data. And I'll do invention disclosures. I'll still do startups with a caveat. And I'll do revenue as much as I hate doing it. But are there for the rest of you? Are there? Are there three to five metrics that you try to look at across the board? For your office? Or do you not do it? Unknown Speaker 42:37 Our I can take this one, our annual when I got to Columbia, our annual report, actually, our quarterly report was about 75 pages of prose, that took about one month of each quarter to write. And our annual report now is a single slide. And it tracks only three things. There's the annual number of licenses and options that were executed. And it shows that all of these numbers are in our track back to 2008. We have one person with absolute and one that's just index, but they're both back to 2008. So it's number of its number of licenses and options, number of inventions submitted and number of startups created using the same definitions. We don't benchmark ourselves or anyone else. I mean, I do have slides that show that that index for research sighs just because some faculty are like, Well, how do you compare to Stanford, and we say, well, you know, here's like, eight or 10 schools, and we're sort of in the range. But our annual report has been boiled down to those three. I recognize, though, that that works at Columbia, to some degree, because it's been a pretty successful operation, both financially and also in terms of keeping faculty happy, and so we're not really under fire. I will also note that our big money license ran out finally, and sort of last year. And so my world needs, I may be going back to those 75 Page quarterly reports fairly soon. But Unknown Speaker 44:14 everybody, maybe we'll never see or and again, Unknown Speaker 44:19 maybe someone else's face is going to pop up in the square next time. In front of in front of low library here. But, but that's ours. Those are the three metrics we track. There's all sorts of other things that we that we track, and actually done to your point. There's all sorts of other metrics that we track internally for our own usage. Like we track how many days has elapsed for MTA days, between when they were received and when we sort of get them off our desk. We track how many days licensed revenue sits on our office before it's distributed out. So we track the amount of active dockets per licensing officer to make sure people aren't drowning in work, but what we report out are those three? That's it. Speaker 2 45:02 John, do you want to comment on what you report out? Unknown Speaker 45:06 Sure we do. Actually, internally, we report lots of stuff but but externally it's it's number of deals, number of companies, how much money they've number of companies that have raised money, how much money they've raised. We also have a pipeline chart of all our products, these, particularly in pharmaceutical and medical devices, by stage of development, from preclinical all the way to market, which we report every single year, it looks like a pharmaceutical pipeline chart. And those are the major metrics because for us, those were, that's where we see the impact. Speaker 2 45:46 So this goes to kind of your comments. So Mark Mondrian, from Virginia Tech says that there is a little bit of analogy in terms of new product development. So not all products launched are successful, obviously. But companies can certainly measure the launches, and they also measure the revenue. And established companies don't necessarily confuse the difference between launches and revenue, obviously, perhaps we have a we can agree that there's a difference between startup creation and startup success. And rather than ask if the autumn definition is correct, the better question is, how do you measure startup success? And I think this is kind of where we're, if I'm sitting here, I feel like we're, we're kind of agreeing that we have just number just creation, we could. And then how do we measure success? And I, yeah, Unknown Speaker 46:40 I think, Oh, Jack, I was actually gonna say a bit Jack has a really good answer for this. Unknown Speaker 46:48 I mean, this this, this was, I mean, this was our quarterly report on on, like, 15. Startups, right. So we, at the clinic, we spent a great deal of time, making sure that we looked at those venture level companies, as a venture capitalist, because we were the LP, we made investments, we're making, you know, 10s of millions of dollars investments in our own portfolio, because we could, but, you know, we, we reported essentially on three different categories of companies, product development, innovator, class and venture class. And the reason we did that was because we wanted to know what the expected outcome was, we could not put a product development company, a company that is going to solve for x with the expectation that we're going to get a return on innovation, by increasing the value of the license, we couldn't put that in front of a venture capital team, I couldn't put it in front of my boss, the Chief Investment Officer, right. So we had to be really careful about how we presented these data, to make sure that we didn't lose credibility. And I think that I think that's what's at risk by not really getting this definition, right. Is reputation really, are you going to lose investors walking in the door, saying, Let me take a look at the companies that you've created? Speaker 2 48:17 Well, I do think that most of us that work with venture kind of know which companies you want to show to certain people, but I agree Unknown Speaker 48:25 on this point, right? It's which Speaker 2 48:26 130 And they all say show us the show us for 30. So I saw I agree, but it doesn't make Unknown Speaker 48:34 that sort of fractal like that's, that's a that's a deep well, you can go down there Jack though, because even if you look at a venture firms, portfolios of things that they've given some seed funds to, there's going to be, you know, there are version like, the wide end of our funnel leads to the narrow end of our funnel, but the narrow end of our funnel would be the wide end of someone else's funnel, and onward and onward and onward. Right. So if you look at I mean, I think like many of us are members of Osage, or UPS, I saw Natasha is on the on the attendee list, but like, oh up tracks all startups coming out of our institutions, and comes up with their own assessment every year, which ones are sort of a B, C group. And, you know, I would actually be really interested in hearing from Osage, I think it's largely a subjective measure. But I find nothing inconsistent saying we launched 25 startups last year. Of those, Mr. President, some of them are just rocking and rolling. And some of them have raised some friends and family. And some of them frankly, are kind of, you know, it's faculty vanity startups. It's an SBIR companies just to get to the next stage in the lab. They're all startups. We're proud of all of them. We're happy that they're all waving the Columbia entrepreneurship flag. Some of them are more promising than others. Unknown Speaker 49:58 But that's the Problem is sorry, Robin, go ahead. You go, I was just gonna say the challenge is that you can control that message, right? You just, you just did a very good job of controlling that message. The challenge is when you can't you get somebody writing an article about the zombie companies that are coming out of university research. And it's based on some data that are not necessarily in line with what we're talking about. Speaker 2 50:27 So I do want to make one comment, because there are, and I'm glad we're going to save all these Q and A's. I don't know that we'll be able to read all the q&a, but I think we are coming to a little bit of this. And certainly the impact metrics committee as well. Have I like what how Oren describes it. Mr. President, Mr. board here, the number of startups that we had, and whether or not you categorize them, and this amount is this whatever what I'm conscious of, because a lot of us are the ones that get involved with reporting into autumn, the thing that always concerns me is metrics creep, where all of a sudden, we're getting a list of okay, give us this list of metrics for your startups? You know, it's one thing to say the metrics are, how much have they raised? And do they have a product on the market? Is there an IPO? Is there an acquisition, but if we start, you know, you can reading some of these q&a, you can imagine a whole other series of of metrics to categorize your startups. And so that's the question, you know, to the panelists, can we get it down to three to five things? Are there? Categories? Do we categorize them by therapeutics, companies? Software companies? Med device? Which actually Oh up does Oh up, does categorize them? By those technology areas? But the question is, do we categorize by technology areas? And then do we have other metrics, but boy, when I'm doing that annual survey, I don't want to have 10 Different things per se, that I'm tracking. Unknown Speaker 52:12 And that's the problem is, yeah, you know, even at a large, all of us work at fairly large offices, and work or worked at fairly large offices and can, and if anyone could theoretically spend all this time tracking and compiling all these stats, it's, it's us. And that sounds like a mountain of work. It's, it's hard enough to even fill in the autumn survey as is. And so the more data you asked for, the less participation you're gonna get. And, again, it's kind of fractal it. Because if we can't even agree on one definition, getting us to agree on 40 definitions seems unlikely. And you got to make sure you're not double counting things. So. So I think it'd be a really great project for the metrics committee to start thinking about what are sort of the five to 10 things that we might want to track? That's not going to break the back of I mean, we've got 200 some odd active portfolio companies now active is, does conversation, some of them are more active than others, but active portfolio companies, constantly tracking each of them would be a nightmare, and frankly, wouldn't help my office's goals. And I probably wouldn't do it. If there were like five to 10 metrics that you want to know. Sure. And maybe Oh, up since Oh, up has, I think is the only firm that I'm aware of that is really had to normalize this across many different universities across many different startups over time. They might be a huge help if you could get you know, Kiersten or, or mark or David or something like that follow up or Natasha to help out. Speaker 2 54:01 Well, another good comment, and this kind of goes to one of my earlier comments was, we're, we're fixated, we're spending a lot of time talking about numbers of companies, and you know, how much funding and stuff and we've have, we lost the university mission of products into the marketplace to serve and benefit the public. And so again, if a company I'm going to have a comp, one of our startups is going to go public, and we will make money off of that, but they terminated the license. So basically, they will never develop the technology, you know, they they pivoted is that a success, you know, so I think those the other the other interesting aspect of Yeah, John, go ahead, you're gonna say something. I, Unknown Speaker 54:52 it's absolutely a success. Companies pivot all the time, but if it started on your technology and raise money, and it went public Like you, it was counted long before. That's true. Yoda counted for Pete's sakes. But I would absolutely say, Hey, this is a great success. In fact, I have done that on other occasions because I, I understand that they pivoted. But but for my technology they probably didn't start. Unknown Speaker 55:20 Right. And actually, I think it's important to think about this question about, have we like, lost something in the questions about like, Have we lost sight of the main objective, which is getting technologies to market? You know, and all this talk about metrics? The reason we're talking about metrics is because we're on a webinar called metrics. like, Nah, none of us, none of us on this, none of the talking heads here are launching startups in order to generate metrics. At least in our case, we're launching startups because it is the only freakin way to get these technologies out of the lab these days. The tech industry will not take voluntary licenses to patents, unless they look they'll only acquire the startups that get the technology through acquiring startups. And largely biopharma would much rather let these things incubate off the books as a startup and then acquire them later or JV with them later. And so. So we are doing startups. And in fact, I would argue what I talked about earlier about why we are doing startups and how we're doing them, we took the let 1000 Flowers bloom approach because what the hell is the point of just filing a patent and sitting on it and waiting for Pfizer to come along license it, it's just not gonna happen. So that's why we, I would rather let that patent go out into the real world and try and survive for a little bit and take it back. If it can survive, then just have it sit on my website. And Unknown Speaker 56:37 you know, jet or a new talk about, the only way strategics will be involved is if it's a company, that's also the only way you can get outside capital to become involved is if they get an equity stake in a company. So I mean, you're kind of stuck, you know, in having to form these companies may be well, before they've got a management team that's robust enough to manage it. Well, before the technology is de risked enough to get, you know, traditional capital to invest, but you're kind of stuck if you're going to keep that technology moving. I, I kind of I'm kind of, I'm going to come back to, I think the metrics are okay. Right. I don't know that they need to be expanded, I just think the purpose of what are you using the money for, you know, what's the category? Is this just an early stage, we're going to try to see if we can de risk it enough to get a strategic involved? Or are we doing it because we need outside capital, and so we have to sell equity in it. I think it's still purpose, more than anything else. That is an important metric there. Speaker 2 57:43 Let's remember, this webinar also started because of this article that was focusing on you know, I see, I believe he ICU. This this webinar that started on an article that focused on two main numbers, and they weren't turning into anything. So that begs the question of, maybe there is a kit, is there a category of startups that we shouldn't count? And or, and I'm going to add, and then John, your next, sorry, or I'll just make a comment. Then Oren can go after John, a number of us do what we call the golden piece licenses for software, where we typically and I know it's not just us, that we do all equity deals. Interestingly, some of those software companies get financing at some point. Some of them go public, some of them done very well, but they're not what you would call the traditional startup. So just to comment, John Europe. So I just Unknown Speaker 58:51 Boren was pointing out the virtues of doing all these things with entrepreneurs. I totally agree with that. We should absolutely do all those things. There is no doubt in my mind that startups are probably the best way of getting things through market. I started off by saying, I love startups, I think they're great, that we should train faculty, we should help encourage students, we should look at it as being an opportunity to advance a technology that otherwise isn't good. The mini flower blue, hell yes. Have at it. Okay, I'm just saying that. I also recognize that some of the just people who without a management team without capital, and haven't done a great market analysis, probably are going to fail, high probability, very high probability. And I would wait until they at least have what most people would recognize as being the three essential elements of a company that has a chance of success before I count, that's all because I you know, and not because I don't have I'm gonna claw technology back or because I just think that it makes sense because we are under a screw Knee with by doll of what impact have you had, not what your numbers are, but what impact and that Nature article and every newspaper article that generates from that, like tech transfer, whatever news or what? Always ask that question. Okay, it's nice, you got that number, but what impact did you really have, and that's where I come from is I would rather side or, or my bias be towards impact, as opposed to just sheer numbers quantity. Speaker 2 1:00:30 So it's four o'clock now. And I had said that I was going to sort of open it to people, you know, raising their hands. So if anybody raises their hands, I think I have the ability to allow you to talk. So if anybody wants to make comments into the group, you know, please go ahead and do that. I'm still sort of watching the q&a, although they're coming up in an odd way. So if anybody's interested in participating as well. You know, please raise your hand. I am looking at I was going to look at some of the other comments that the the impact metrics people had been mentioned. I think that the other question becomes up someone's raise their hand, you know what, I am just going to let, let's see if I can do this. Unknown Speaker 1:01:24 Okay. You are up? Unknown Speaker 1:01:29 Happy, happy to be a guinea pig. Can you hear me? Unknown Speaker 1:01:31 Yes, we can. Okay, Unknown Speaker 1:01:33 so I have a question kind of directed to John about measuring impact. You know, the scrutiny that we face because of bipedal is national, but CTOs operate locally? So my question is, do you think that smaller GTOs in less vibrant economies are going to suffer if we start to base our metrics on the national scrutiny? The bipedal faces? Unknown Speaker 1:02:03 To answer that question, so the definition of flyover state is anything that is west of Cambridge, Massachusetts, and east of the 101 running through Palo Alto or next to Palo Alto, with the possible exception of orange and Columbia, the rest of us are all flyovers, dude. It doesn't matter how big you are, how small you are. But my The thing is, I think that you can talk about even in a small startup, if it was out in the Midwest, and it was a public university, and they were doing, you know, reporting numbers that they've done five or six startups, somebody's going to ask the question, where are they? Okay, you did five companies last year. This is fantastic. Where are they? Show me the human beings show me the people who are actually executing against that. It's a fair question to ask all of us. And, frankly, just so we're clear about this, the first couple of years, when I was at Yale, I didn't go back and read all the different reports that I because I knew how many companies that started. And my first year was zero. My second year was zero. My third year was one. You know, because we recognize that takes time to get these things started. But we wanted to report. Now the interesting thing is the first six startups, five of them, actually made it through to be acquired, one of them failed. But the other five were acquired and their products came to market. It was great. So Unknown Speaker 1:03:37 actually, I think this is a really interesting, this is a really interesting question. And that's sort of what I was trying to get out earlier. John, you know, Yale's actually, I think it'd be actually really interesting to look at the universities based on how many of their startups have had achieved sort of massive success, whatever we define that as the could be a big acquisition or a big investment round, or, you know, it gets us down this whole other rabbit hole, but, but I would argue that Yale probably has had higher quality startups than 97% 98% 99% Are there. So universities, like you've done an outstanding job of getting those and for your needs, and your President and your provost and whatever, you know, you faced in New Haven, or from the Governor, that actually probably fits your needs quite well. Like for Yale doesn't need to brag about the 10 little tiny startups the like new iPhone app or the you know, the small little diagnostic kit for something that, you know, for home use for record, you know, for like athletic performance athletes trying to train at home like this kind of stuff may does not rise to the level of brandable at Yale. But there are lots of places across the country and taken lots of universities where the local newspapers or the university's press office will actually write articles about some student who launches a company and they're proud of that. Even if the company goes nowhere. And so I guess what what makes me nervous, and I think maybe we're Mitchell is going is different universities need to brag about different things and have different pressures on them. And I think it's awesome that, you know, a small university or college in a town, that is where the economy is dying, can brag about those kinds of things, if that's what's important to them. And if someone said, Show me the money, like, where's the company? And you said, look, it's it's someone in a dorm room for now. But they're dreaming big like that university, you might decide that still breakable. And I would hate for us as an industry to shut that down. Speaker 2 1:05:40 So this kind of begs the question, even listening to just now, should we even report number of startups as an autumn thing? Certainly, in our own ecosystems we can have the way we count them the way we the way we count them that they started, and also the way we measure success for them. But is it? Is it an impossibility to have a US one? A, you know, us wide metric for startups? That's just a one number thing? Unknown Speaker 1:06:19 I don't think that would make any more that is what we do. And I would argue at one number, but you know, but our stage of maturity is an industry, I would think that would make no more sense than the federal government reporting number of companies in the US and counting likes hotdog stand alongside apple. Speaker 2 1:06:37 Well, wait a minute, but but again, maybe you didn't hear my question, Should we just stop the autumn survey asking for number of startups? Because what you're what you just said is kind of what I'm saying. Yeah. Unknown Speaker 1:06:51 So you could do that, although I actually think it'd be perfectly reasonable to have a start splitting like, Okay, how many startups overall? And then of those startups? How many have raised at least half a million dollars, I'm making these numbers up half million dollars of professional funding or generated half a million dollars of revenue? How many have gotten to some other stage? How many have a product in clinical trials? How many have have sold a product or service? I'm, I'm you could 1000 different? I mean, Jack, I don't know what you used in that report you were showing, but like, you probably have some stages, or Unknown Speaker 1:07:27 how I guess it was it was gonna be my point is that, you know, it for some companies, it doesn't matter how much capital they raised. It just matters that they advanced the technology. And for other companies, it matters a lot that they got to their be round. Otherwise, we were not going to follow on with funding. So I don't think there's any, any normal type of metric, I Unknown Speaker 1:07:50 think, again, Unknown Speaker 1:07:52 I think all the metrics are good. I just think that it matters, you know, what is the or, and that was kind of funny that you just dropped down there. I I think what matters very much is having, you know, the use of funds being probably the driver for it, because that's always John's problem as well, John, you know, saying he doesn't want to count a start up until they've got outside capital and an honest to god management team. Okay, that that's fine. But those are metrics that are included already is do they have management team? And do they have funding? Right? Yeah. And I guess that's Unknown Speaker 1:08:31 not the definition of startups. Unknown Speaker 1:08:35 Just a minute. The money part is not part of the Speaker 2 1:08:41 Right, right. So just as a reminder, and I could if I was really slick, I could figure out a way to get the definition back up in front of the screen. But the autumn definition, again, we put it on the PowerPoint is literally did the company form? Well, Ivan did the company form based on a license from the university has nothing to do with doesn't have any management? It doesn't have anything to do with whether or not it got any money? It actually, and I guess I know, some people don't even interpret it that it actually had to have a license, it could have an option. So that's it's a very broad definition. Unknown Speaker 1:09:23 But what if you what if you took that and I think this would get maybe Hannah from Cal Tech said this in the comments, but what have you had are? I mean, you could first of all, you could just kill our definition because it is to some degree a little meaningless. And and it is used largely for the arms race, and I mean, published the way it is. It's a bit of a mishmash of different things. And it largely contributes to these ridiculous reports. No offense, the Milton report, if anyone from Milken is on here, we love the Milken report, but but it does leaves these ridiculous rankings that come out of nowhere. I'm so killing, it's maybe not a terrible idea. But but if you said like, how many of the universities? How many, how many companies that have how many startups that have licensed, licensed or option University technologies raised their first x this year, or first brought our product to market this year or first entered clinical trials this year? Because otherwise, you've got a little bit of a longitudinal problem. Otherwise, we're counting things when we execute a deal. But then what if they raised their money or go into clinical trials two years later? When do you count them? So but I don't see anything wrong with having four or five? If it was four or five metrics, we can keep track? I would think. Speaker 2 1:10:42 So that kind of, again, I hope that impact folks are listening, you know, you could have instead of counting on that is distracting you just by my power cord keeps falling out. And I keep losing it. Sorry about that almost begs the question, could we have something instead of counting startups? When we did the license, having these count, when they have a product when they have an IPO when they have an acquisition? When that? Again, we're getting into, you know, this metric creep and how difficult it is to track these things. It might work? If, if the definitions were were very clear, and this is always the issue, Unknown Speaker 1:11:33 learn all those data collected anyway for economic impact, Unknown Speaker 1:11:36 not by Autumn. And not by by the TCO. Unknown Speaker 1:11:41 I mean, we don't for instance, Speaker 2 1:11:43 at Columbia, but we don't. Well, you don't you? You you track how much they've raised, right. No, Unknown Speaker 1:11:52 good. We don't. Speaker 2 1:11:55 So actually, I'm going to we I, we have people that want to talk so I'm going to allow them to talk. Unknown Speaker 1:12:03 Can we do a poll though, Robin of the people on? Does anyone, Barbara, Holly, do you want to do a poll? I'd be curious, what percentage of the people on this call and reliably track how much money their startups are raised? I would think, a good chunk, but I'm not sure. Speaker 2 1:12:22 So we've got two people that want to talk. We've only got about 15 minutes. So let's let's get so what is Doug go for it? Unknown Speaker 1:12:30 Just a quick comment. I would think that we should continue to count the total number the way it is counted, because it's it's got a lot of history. I agree that there, I thought it was clear that you could only count a license. We've never counted options for a startup. But anyway, separately, starting to count some kind of success metrics. Pick the number that success metrics that you believe, John, you're the ones that you say, Yeah, this is now a startup. I think you might actually start to affect the arms race. Because if people are doing 50 startups and to success metrics, or one to two startups in the success, they will look silly. And so they will start to think about more carefully what they're launching when they're launching it. And so those numbers get to be realistic. That's my thought. Speaker 2 1:13:29 Okay, next person near and dear to me. Rob Helfer. Unknown Speaker 1:13:38 Obviously, she's only picking people she knows. Now Speaker 2 1:13:40 these are the two people that raise their hand. They both happen to have worked with me at some point in their life. Unknown Speaker 1:13:47 Yeah, longtime listener first time caller. Thanks for Thanks for taking. Yeah, I wanted to go one beyond what or instead, you know, for schools that don't have new venture support programs, all startups are effectively lottery tickets, you know, that the once the license is done, the outcome is out of their control. And you know, their next interaction is likely a breach letter. So I think counting startups using the autumn definition is probably the right thing to continue to do given the history. I think adding metrics, they end up being more like revenue in the sense that they're really money raised and products to market are lagging indicators of things that you did five to 10 years ago. And we saw that with two IPOs that we had last year. So I think a nice mix of sort of what you're doing now and what you you know, what is the outcome of what you've done over the over time is is a reasonable solution here. I'm gonna hang up and listen, thanks. Barb, did Speaker 2 1:14:47 anybody figure out if we could do a survey? I don't think we can do a survey. Maybe we do that? I do think it would be interesting. We are tracking the poll in progress. Okay. Do you do your live We track how much money your startups have raised. Right now, it's, well, like all polls, maybe we should wait, we'll have to wait until the maylands are in because it's it's changing rapidly. So let's see. I guess the other question, could we have a poll that says, are people interested in a subcommittee? Or how would people and I guess we have to work? Want to work with the impact committee? On this? I mean, there's a piece of me that wants to ask the question, should we should autumn measure number of startups? You know, or do we change how we how we measure it? Unknown Speaker 1:15:48 Wait, but I'm robbing for that poll. Is it is the question, should we maybe should we only measure number of startups? Or should we measure a number of startups plus some other metrics? Or should we not track number of startups at all? Speaker 2 1:16:04 But Can people see the results of the poll? Or is it just me and I need to read the results to you. Okay, so right now we have about 65%? Well, they're changing again. But interestingly, the, the question was, do you reliably track how much money your startups have raised? And it's, it's interesting 41% Right now say yes, 30%? No. And other track, but not reliably is about 30%. So it's all over the map? In this, so it's not a clear majority. So I guess, I don't know, Barbara, or Barbara. Holly is doing the poll, we've got about 15 minutes. Is it worth asking? The question, Should autumn measure number of startups? And then I suppose we could have a corollary that says, should start up speed measured in a different way? And I my view would be if we say should, should autumn measure startups, and you get it? No, I'd argue then the next question be, should we measure them in categories? Or something? I don't know an Oran, have you got a better way of counting? Unknown Speaker 1:17:28 I mean, if we want to take a poll, I would just I would, I would have three, like, Which of the following best describes your point of view, basically, like, auditor should not track this metric at all. Autumn should keep doing exactly what it's been doing. Or autumn should track the metric it's been doing but add some other categories as well. Maybe that's it, Speaker 2 1:17:49 stay the same. Don't track it at all something different. And Unknown Speaker 1:17:54 by the way, Robin is if anyone I don't know, barber, Holly, if anyone from the metrics committee is on, maybe they could grab the floor for a minute or two and just explain. Like what the metrics committee like what it is and how people can get involved if they're interested in getting involved. Speaker 2 1:18:13 I thought that I'm not seeing a raised hand yet. So. Okay, so we do have the poll open for the people that are still remaining. The questions are up. So you know, should we measure should we keep doing what we're doing? And add new categories? So let's give that poll a little bit of time. To see how this how this goes. I don't think we get to vote. Is this something peculiar on? Unknown Speaker 1:18:50 I just voted, but did you Unknown Speaker 1:18:53 read it twice? Rather than being Unknown Speaker 1:18:54 so open on my Unknown Speaker 1:18:57 book? I just got it. Oh, I Speaker 2 1:18:59 see it. Sorry. So interestingly, there shouldn't be a surprise to anybody. We've got just about 65% of people who voted. Should autumn measure number of startups autumn should not track startup 6% Autumn should keep doing what it's doing. 17% Autumn should track what it's been doing, but add new categories. 78% Survey says. You know, I think if anything, and thank you to all the attendees, we should have thought about these poll questions earlier. You know, if anything, there's a result and then the next steps. How do we how do we figure this out? Unknown Speaker 1:19:52 You know, Robin, one thing that we didn't get into in a couple of minutes left. I think that just from a reputational perspective or in In this point about the, the the way to handle, you know, the nine points and reporting bullet I can't remember, nine points to consider for Responsible metrics. Well, I just I mean, I know that that's kind of part of what that poll was just about. But at the same time, you know, it feels like the group should have a louder voice in the leadership and, you know, getting out in front of documents like that one that I'm not too fond of. Speaker 2 1:20:33 And I also want to note in the q&a section for all of you who are participating, we are going to keep track of all the q&a and Joanna, somebody's mentioned, Joanne McMaster mentioned, and I think this is a good idea. anybody on the call, if you want to share any additional requests, metrics, comments about the impact metrics, you know, specifically related to this, go ahead and put them in the q&a, because that's one way that we can track them going forward. Because it seems to me, the next step is to, you know, gotta help us. May we have an autumn meeting in real life where we can actually have a real discussion about this. But but please feel free to send those those comments in that way. Of course, you can always send them to autumn or to any of the knuckleheads that are on the panelists right now. Let's see. I haven't seen any other. Any other hands up. Doug just sent me. Yeah. Yeah. So any other comments from the panel? It sounds like, we're coming to the point where we need to come up with some other metrics, or some other consensus. Unknown Speaker 1:22:00 What do you think? But not on this call? Presumably, you Speaker 2 1:22:04 don't know. I don't think in 10 minutes? I don't think that's possible. Unknown Speaker 1:22:10 Yeah. So I mean, it would be great Barb and Holly, it would be really great if someone from Autumn could could if someone could put in the chat, perhaps the contact information for anyone on the metrics committee? Because honestly, I don't know if any of us on this phone call actually want to be collating all of those responses. So if people want to join the metrics committee or or have thoughts for them, Is there is there maybe do you guys have a contact information for anyone on the metrics committee? Speaker 2 1:22:44 Well, I think I have but certainly the autumn, folks, do Unknown Speaker 1:22:48 you want me to? Unknown Speaker 1:22:50 Yes, we will. If you want to just put something in the chat about wanting to either volunteer on our committee or if you want to contact the committee, you know, please put it in? I can. Regan Robertson is I believe the chair. And I can quickly try to find his contact information and put it in here, but he might not appreciate that. So let us know. And we'll pass on the information to him for sure. Speaker 2 1:23:17 So absent anybody having any other comments, we can give you seven minutes left of your life that you thought you were going to have today. But I do i We did. I want to thank everybody for sending in comments and and putting into q&a. Those of us who love metrics, and I have a love hate relationship with them. I'm excited that this will this discussion will continue. And maybe in a year, we can have a discussion on how we track agreements. My other nightmare. So Speaker 1 1:24:01 anybody else have any parting thoughts before I close this out? Okay, on behalf of autumn, I want to thank Robin or and Jack and John for this great discussion. And we want to thank all of you for attending. We appreciate your comments. We appreciate participating in the polls. And we hope you found some information here. If you do want to get involved in the metrics committee, I don't know that we have I don't think we have the the contact information. But you can either respond to the email that was sent out with a link for today. And I'll make sure it gets to the right person. Or if you put it in the chat or q&a Right here, we'll make sure it gets to the right person. We have recorded this today. So we will have a recording available within a few days. And when you close out and evaluation will pop up and we'd really appreciate it if you could fill that out. And that concludes our program today. Thanks for joining us. Have a fantastic afternoon. Transcribed by https://otter.ai