web design software quora

web design software quora

>> good afternoon. today we have two different speakers. we have stewart the founder andceo of slack and then we have adam,the founder and ceo of quora. i\'m going to interview stewart. we\'re going to talk about ideas and how tofind good ones, and then adam is going to talk about metrics why they are important,and what to measure. stewart, thank you for coming andspending some time with us. stewart is the founder of flickr andslack, and what i didn\'t know until very

recently is that he tried to bethe founder twice of massively multiplayer online games, and they turnedinto these two iconic companies. so i wanted to talk about just the processof coming up with the idea for the game and then the ideas forflickr, and slack. >> so yes, twice we set out trying tobuild a web-based massively multiplayer game and the origin of that was all theway back in 1992, i arrive at college and i got my account on the school\'sunix machine and was exposed to the internet for the first time andit was kind of a mind blowing experience. so, this is about a yearbefore the web became popular.

so it was like irc, hughesnet, talk, emailand i was really fascinated with the idea that people could find their communityirrespective of what that community was. so if you were a breast cancer survivor ora model train enthusiast or whatever, if you were interested in some as iwas obscure 19th century biologists, then you can find all that stuff. and the idea of a game that lived onlinelike that and had the game like aspects or play as a pretext for socialinteractions seemed really interesting. and i\'m not going to go into tolong of an explanation, because it turns out it\'s like a terrible, terribleidea at least as a commercial business.

like it might have been fun as a hobby,i think it could have been the kind of business that sustained the numberof employees required to operate it. so, like a restaurant orsomething like that. but when in 2009, we startedthe company that ended up making slack, we raised $17.5 million in extracapital and it\'s just never going to become the kind of businessthat would justify that investment. the first time was a little bit different. so, 2002 was after the dot.com crash. it was after worldcom andenron accounting scandals.

it was after 9/11 andthe nasdaq was down 80%. s&p 500 was down, 65% from its peak. no one wanted to investin internet facing stuff. so it was very hard to tell whetherthis was a good idea or not, because there was no signalfrom the market that said, this is an investable idea, becausenothing was investable at that time. we raised a little bit of friends andfamily money. we spent about a year orso building a prototype. the prototype was well-received, butit was going to take us another year and

a half to finish it andwe couldn\'t raise any money at all. we got to the point where the one personon the team who had kids against was the only person that\'s getting paid andthen we decided to try to find something else we could do that would make use ofthe technology that we already developed, but that will be quicker to get to market. and that thing was flicker, butit was also a bad idea at first. so because we had developed this gameclient and we had a messaging server on the back-end, we just re-purposed the gameclient as a photo sharing client. so in the game you had a bunch of slotsalong the bottom of the screen for

inventory, those became slotsinto which you can fit photos. and because the game wasall real time interaction, the photo sharing was all real-timeinteraction which meant that you have to be online at the same time as someoneyou cared about in order to use it. and while it demoed really well,it was amazing technology, especially for at the time we started andthat maybe like late 2003. it was a terrible product, because you didn\'t care about any ofthe people who were actually online. that morphed really quickly and

the experience making slackwas a little bit different. we had all of the resourceswe could ever ask for. i mean, we had more than enough money,because we had money leftover and we decided to abandon the idea andwe had aws which we didn\'t have before. hardware was effectively free internetconnections were much better. there were many more people online. hardware was better. so, none of like those externalconditions was the reason this wasn\'t. this massively multiplayer gameon the web wasn\'t a good idea.

it just wasn\'t a good idea, becausethere wasn\'t a big enough market for it. >> and why was slack a good idea and how did you decide thatslack was a thing to build? >> well,we started developing the system for internal communication that was base ircand those of you who had irc kind of, and also use slack can recognizethat slack is irc with all of things that were missing when it wasoriginally designed in 1989. i think this is, i don\'t know,it\'s like it\'s post facto analysis. so it\'s hard to know if thisis really true or not, but

i think that the reason this lock has beenso successful is because we had a 3.5-year long design process where we weren\'tconscious that we were designing it. so, we started using irc. irc has many deficiencies. among them, you can\'t store andforward the messages. so if i wanted to send you a message andyou\'re not connected to the server that the moment i want to send it,i just can\'t send you a message. so, the first thing we built wasabout to log all the messages and put them in our archive.

and once we had that,we had messages at a database and we thought search would be useful, sowe built search for it and this was 2009. there were no good irc clients foriphones. and so, we built an html5 front-end forthe archives and safari. and once we had that, we wanted to be ableto post and just like on and on and on. but in the normal processof software development, there\'s a huge amount of ego andspeculation. so when you\'re arguing with someone aboutwhether this feature is a good idea or not and if you can get to the pointof enlightenment and a buddha sense,

then maybe you can dothat without any ego. but the practical reality is people arevery ego laden in those discussions and there\'s a lot of speculation like, i thinkthis could be really valuable for someone. whereas when we were building the systemthat was kind of the proto slack, that was the inspiration for slack, it waswhatever problem is the most irritating. like something that we just absolutelycan\'t stand, we would address that and then with a minimum amount of effort andthen go back to what we\'re supposed to be doing and then kind of let cook forseveral months. so we had 45 people working on it,working on the game.

so, we had a lot of real world feedback orwhen there was a problem. i mean, when an opportunity that was soobvious that we couldn\'t help, but take advantage of it, we would spendthe minimum number of minutes on that and go back to it. so the end of that process, we had thisfully designed product that we were using that was a terrible implementation notwhat you would do if you started from scratch, but it was obvious that thiswas something that had enormous value. everyone who worked at the company agreedthey would never work without something like this again.

and so just on that basis,it seemed like it was a great idea and we had years of evidence. >> were there other thingsyou considered pivoting to or was it just like there\'s this thingwe\'re going to do it, it\'s working. >> i wish that i had better access to theproposals that i personally had written, ciz i had a whole bunch of **** terrible,terrible ideas and this was the one that had the broader consensus among the teamas being something that would be valuable. what\'s interesting at the time though,so we had $5 million left in the bank. none of our investorswanted their money back.

they want us to just gotry something else and jason horowitz is one of our investors,so we went down and did the partner meeting andkind of presented what our big idea was. and at that time, this is before weactually started development of slack. but when we had this proto version,we said that one day, this could be like a hundred milliondollar a year revenue business. it could be valued at a billion dollarson that basis and we thought that, that was like a decade out andthat would be the culmination. that would be we have 100% of the ten and

we went past that in year two,kind of blew past it. we\'ll do 200 million ingap revenue this year and still be growing at a very fast rate. and now we think that number ismore like 10 billion in revenue. like a hundred billion dollar company. so we didn\'t realize howgood of an idea it was, but it\'s also there\'s an interestingstory about deciding to shut it down. because we still had money, because we hada relatively big user base who were very engaged, but we had a really like a leakybucket, it was very difficult to add

new users to the game that we\'reworking on, which is called glitch. i felt very stuck in this position, because on the one hand you have narrativeof good entrepeneurs are resilient. when everyone else thinks it\'s a bad idea,it\'s probably a really good idea. you have to dig in, prove people wrong. keep going, even when times are dark. one the other hand,there\'s a morning when i woke up and it\'s just like,i can\'t do this anymore, i\'m the ceo, i\'m the chairman of the board,i\'m sure that this is not going to work.

and it\'s actually hard toconvince everyone else. >> i think that\'s one of the hardestthings about evaluating ideas. is saying when thisone\'s not going to work. so, do you remember what it was aboutthat morning when you woke up and said, you know what, it\'s time to move on? >> i don\'t.i mean, i think people have very little interest in access to their owncognitive processes to begin with. so anything say would bekind of making it up. but i think it was i had exhausted allthe ideas we had to fix problems, so

we focused on new user experience. and we really wanted to teachpeople what this was and kind of what the possibilities were andget them excited about it. because we were okay at attractingpeople into the top of the funnel but had a hard time retaining them. and. >> that\'s always when i tell people togive up an idea when you\'re out of ideas about how to make it work. >> yeah exactly, so we\'d have an idea,try it didn\'t work, have another idea and

at first you can fill up post it notes,like those giant boards and you can fill up white boards with ideas and you argueabout which ones are the highest value. and then one by one you eliminate them. months go by, a year goes by,another six months goes by. and it\'s like ****, these are all of theideas we thought were the worst ones to begin with and we only have three left sothis is probably not going to work. >> you mentioned at one pointyou had a bunch of proposals, how do you come up with ideas? or how did you even think that you know,

slack is maybe this thingis a really good idea. how do you come up with new ideas andhow do you evaluate them? >> coming up with new ideas is easy. like i just i feel like it\'s hardalmost to not come up with ideas. i mean,they\'re not necessarily good ideas so the recognition of which onesare good is actually much harder. i recently finally got aroundto reading thinking fast and slow and there\'s this story there ofa psychologist named herbert simon who studied in chess grand mastersbecause chess grand masters

are one of the category of peoplelike really experienced doctors. that people regard as sort of miraculousin their ability to intuit a situation, like glance at a board and say,white checkmate in three moves. and people thought that this was likesome super well developed intuition. and professor simon argues that it\'s justa recognition in the same way that like a 18 month old can look at a dog andsay doggie. or within the first 500 milliseconds ofsomeone speaking you can tell if they\'re angry at you, like you just developthe ability to recognize situation and ques which.

are hard to disambiguate, they\'re hard toarticulate and enumerate in your mind and say exactly why you think this way. but over time, you get better andbetter doing it. i\'ve been making softwareprofessionally for about 20 years, so the specific micro-ideas, or sub-ideas, about feature implementation, or the orderof actions in the user experience. or whether a certain feature is moreimportant than another feature. those ones,i feel like i have an intuitive sense. which is actually just the abilityto recognize, over time.

so it\'s really justa process of filtering. like you just have constant ideas. everyone has ideas. we at slack have developed a lotof organizational design and practices to try toabsorb as many as we can. so we get about, somewhere between 10 and20,000 tweets at us a month. and all of those are respondedto by a actual human being. we get a similar number of customersupport tickets, and we have processes for trying to filter up and abstract allof the feedback that we\'re getting.

but even down to the levelof all the tweets at slack the company are pumped into ourslack instance and people check them, and they can add an emoji reaction andthat filters it to another channel. for the ones that seem above somethreshold of this seems like a good idea, and it also seems practical. i could imagine that we could do it. and then there\'s some debate atthat level, but, i mean, every well, because almost every person i knowin the technology industry also use slack. every time i talk to any ofthem they have ideas for me.

i mean they have complains or proposals>> so the big ideas orthe little ideas i get the intuition on. what about the big idea, did you spenda lot of time thinking before you decided to go ahead with slack, did youspend a lot of time thinking about this would be really strategically valuable andthere would be a deep network effect? >> no not at all. i mean so we thought that this wouldbe valuable for individual teams and we could see that valuedemonstrated in our own experience. we definitely didn\'t seehow applicable it would be.

because right now, we have 38,000paid customers around the world, so that\'s a lot. we have a sales team but we\'ve nevertalked to 90 plus percent of our customers because we just couldn\'ttalk to that many people. so again, this is already farther thani thought we would get at the very end. and so we under recognize it. those big ideas are, i don\'t know, we talked briefly about this topicbefore and one of my favorite lines is, i have a great idea for a novel all ineed is for someone to write it for me.

because obviously preposterous. right?like no one would really think, i mean only a really deluded personwould think that that\'s true. but people definitely have ideas for apps all the time, andthose can be harder to assess. and it\'s so rare that i hear one wheresomeone proposes an idea where it just, it\'s like prima facie that is a goodidea as opposed to [sound] i don\'t know, i mean it really depends on the execution. actually just last night,i was watching something on youtube, and

the ad that came up before was for an appcalled mileiq, and it\'s you turn it on, ask for location permissions, andthen every time you go on a car ride, it just looks to see the speedthat you\'re traveling. and then at the point you stop,it stops it. so kind of like an automatic strava. like it just records all your trips. and then you can swipe right to say that this is a trip that my company\'s supposedto reimburse me for and swipe left, not. anyway, it\'s a very simple idea.

it\'s actually a really great idea becausepeople have trouble recording those, they have trouble in audits with the irsto prove that they used the card for this purpose and stuff like that. and that\'s one that actually seems like. hm, the execution doesn\'tseem very difficult. i think that would be a good idea. if you could market it effectively, itwould probably be a successful business. but like 95% of the ideas i hear,i have no idea whether it\'d be successful. >> where are you generally onthe spectrum of ideas matter and

it\'s worth the effort tothink of a good one and i think it\'s just all that execution,and ideas don\'t matter at all. and you just have to keeptrying things until someday one of your internaltools turns into slack. >> so there\'s, i\'m there\'s a lot of people in this class and thenthere\'s a lot of people at stanford, and there\'s a lot of people in south bay andthere\'s a lot of people in the bay area and many many people want to start techcompanies and all hope to be successful.

so to a certain extent there is like 100million monkeys banging on typewriters and someone\'s going to start a shakespearesonnet so, there is a little bit of that. like a little bit of survivorship bias and other cognitive biasesthat come into play. on the other hand, i don\'t thinkeither end of the spectrums is right. i would definitely leanmore towards execution. there\'s a good steve jobs elaborationof john sculley got hired and started hiring all these people at applewho thought that only the ideas matter. the idea absolutely matters, but

the idea when you start hitting executionhas to accommodate so many things. in steve jobs case, it\'s about whatit\'s possible to get glass to do and how small you can make batteries and whatkind of tensile strength plastics have, and stuff like that. because your idea has to stand up to allthese things and it\'ll transmogrify and alter and may even be unrecognizableat the end, but if you\'re really good at execution and the core of the ideacan make it through all those steps. so i absolutely favor execution,but you can execute a lot on a terrible idea and then notget anywhere so you kind of need both.

>> if you were startinga new company today, you know april 2017,where would you look for ideas? what sorts of things wouldyou be thinking about? >> i don\'t know if this works forother people, but i would definitely look at my ownexperiences as a consumer, generally. because it\'s very easy forme to see things that are frustrating, i complain about stuff a lot like,especially given how good my life is. my life is really good and yetevery time i have to fly somewhere i have at least ten significant complaints and50 minor complaints.

and every one of those is an opportunity,right? every one of those thingsthat don\'t go as well as you would like are opportunities forimprovement and some of those seem like someone else should see them andit should be really obvious. but they\'re often not. there\'s this story i tell in ourinternal on boarding process of me and our head of product design going fora walk in vancouver. and our office in vancouver is ina neighborhood that has really narrow sidewalks.

the sidewalks have sandwichboards from vendors out so it\'s very crowded soyou kind of wind around. and it starts rainingwhile we\'re on this walk. and maybe two thirds of the people hadumbrellas with them and we didn\'t and people are walking towardsus on the sidewalk. and almost no one would movetheir umbrella out of the way so that the pokey thingswouldn\'t get us in the eye. and like i said, the sidewalks are verynarrow, so we had to keep on ducking. and i can come up with manyexplanations for why this would be.

but don\'t explain with malice thatwhich can be explained by ignorance. so it probably wasn\'t just thatthese people have few avenues for exercising power in their lives, and they were choosing like thismoment to exercise some power. but then there\'s onlyreally two explanations. one is,they just walk through the world and they don\'t see that we\'re going likethis to get away from their umbrella, even though inevitably they\'vehad that experience themselves. or they see that this is happening andthey\'re like,

i just can\'t think of anything ican do to ameliorate the situation, despite the fact that it\'s this tiny,like a hundredth of a calorie, worth of effort anda tiny amount of consideration. and the point of telling that story isthat that\'s the way, and this is a sad way of looking at the world, but that\'s theway that most people go through the world. they\'re oblivious to the problems thatother people have and if they notice the problems, they\'re unable tocome up with any kind of solution. so like two thirds of the peoplejust didn\'t tilt their umbrellas. which means that if you\'re the kindof person who\'s willing to tilt your

umbrella, there\'s a whole worldof opportunity out there. >> do you have any tips on how to noticethat you should tilt your umbrella? >> pay attention. i mean, that is the biggest thing. but pay attentionespecially to other people. so you can pay attentionto your own signals and maybe if you get practiced->> this has actually worked at the [inaudible] history when peoplehave needed to come up with a new idea on the fly, it doesn\'t usually work,but when they really just commit to

paying attention to other people and sortof what seems broken, that has worked. and then so if you do it this way youcome up with like thousands of ideas. anything you\'ve noticed as a good way tofilter those besides just building up the experience of seeing a lot andthe recognition? >> yeah there is the kind ofdialectic between execution and idea because there\'s definitelysomething to be said when things are above some thresholdof good to sticking with the same idea. not so much because that idea\'sthe best one, but because the wild and erratic chasing of the next idea

when you have something that has a littlebit of traction can be really dangerous. so, slack is a good example of that. we conceived of slack as a tool forbusiness. we developed it that way,we priced it that way, it was successful. but, there\'s also this hugeamount of social usage, like now, maybe a third of the slack teams that getcreated are for some kind of social usage. and a lot of people had the ideathat we should pursue that, we should make slack for groups, or evenjust pivot to being a consumer company, like kind of a private version oftwitter or something like that.

and we\'ve been pretty steadfast in notdoing that because you can only do so many things well. and there\'s a lot of shiny objectslike that, that are very distracted. >> are you the one that says noto most of those new ideas or you have you had to buildthat into the company? when something\'s going as well as slack,there\'s so many new ideas. >> i\'m chief no sayer but there\'s definitely a whole bunch ofpeople who are equally committed to no. >> great.we can probably take one question for

stewart if someone hasa really good one before- >> only a really, really good one. >> maybe that was somuch pressure now no one\'s going to ask. >> [laugh]>> so you already had a successfulcompany before slack. i was wondering what the differencesbetween as a first time founder and a second time founder? >> what are the differencesas a second time founder? >> so everyone of the externalfactors was much easier.

so the first time i mentioned thatwe ended up making flickr because we couldn\'t raise any money and we\'redesperate to try to finish something. fast forward seven years, andwe could raise as much money as we wanted. so that was easy. we didn\'t even make a dac, we just said,we would like some money and then people gave us money. so a lot of those things were easier,it\'s always easier to get press, it\'s easier get attention,easier to recruit, easier to raise money. it\'s not any easier to be successfulcertainly because we started the company

that made slack in 2009 and failedpretty hard for three and a half years. kind of like pushing intothe wind of something that wasn\'t ultimately going to work. and we could have just stopped there. it was great forinvestors that we didn\'t recap and we just kept the same company and sothat company has now been successful. we could\'ve also just walkedaway from that company and incorporated a new company. and then it would\'ve been my secondcompany was a failure instead of

my second company was a great pivot. i\'m not sure which things get harder,i\'m not sure than any do other than the degree of self criticism andlike awareness of your own limitations. great, thank you very much. >> no problem. >> appreciate you coming in. >> [applause]>> now, we\'ll have adam talk about metrics. >> great, so thanks for having me.

i\'m happy to be here. i\'m going to talk about metrics andmeasurement and this whole group of issues that comeup as you\'re trying to start a company. and, one of the things, i think is reallyinteresting about this topic that is so much of what you have to do ina startup is specific to the startup. you\'re going to have to get tobe an expert at some domain. you\'re going to have to get to bebetter than anyone else at that domain. and a lot of what it takes to do that is, hard to teach in a class because it\'sjust different for every startup.

but, metrics is this domainthat\'s kind of general, general enough to be usefulto almost all startups. so, i\'m going to startoff with this example. so, this is the first version ofamazon.com from the early 90s. [cough] and,i really like looking at these. you can look at the first versionsof lots of different products. it\'s a good reminder that it\'s mucheasier to get these things started than you\'d think. even these companies that are massivetoday, if you just go and

look at how they started,this is a good way to keep yourself in touch withthe reality of how companies get started. so, does anyone know why amazondecided to start with books? yeah? >> choice. >> choice, yeah, yeah, yeah, sothat\'s actually, i think what you mean is that you have a lot of, you\'ve got tochoose out from a lot of different books. yeah, yeah, soit wasn\'t just that they chose books, it was that->> [laugh]

>> it was books give you, so, jeff bezos wanted to a start a store andone of the most important things when you\'re gettingstarted is differentiation. so, you don\'t have to make a productthat\'s going to appeal to everyone immediately, but, you need to makea product that is going to appeal to some people more than anything else. so, you want to find some wayto differentiate, and usually, you want to do that by doing somethingthat\'s going to be easy for you but that\'s really hard forall the existing companies.

and, the internet enabled this inamazon\'s case because brick and mortar stores have limitedspace to hold their inventory. and so,you realize that as on the internet, you can have a big warehouse somewhere,you can have items shipped from all over. this is something that enables you to offer much wider selectionthan a normal store. so then, you go andrun through the thought experiment. you say, well, if what we can do isprovide better selection, then which category, what kind of store is going tobenefit the most from a greater selection?

and it turns out that books isa category where you have the most, there are more books than thereare other products, right? so if you look at like movies,it\'s very expensive to make a movie. that\'s not a industry thatwould benefit immediately from the internet in the way that books would. so, i think there\'s a generalpoint here that measurement can turn a vague idea into a good idea. and, if you do a lot of these kind oflike back of the envelope calculations, it can really helps sharpen your ideas andit can help you iterate and

move through the space of ideas a lotfaster than if you have to go and like, build the prototype. you can measure what amazon started andthey started to focus on movies and after a few years, they realizedthat it wasn\'t working that well, then they\'re set back by a few years,now then there\'s competition. these kind of metrics on, even in the realm of ideas canreally help you get started. and the one other thing i\'d say about thisis that, i think using measurement here is very good for helping you figure outwhat products are going to be possible or

where you\'re going to be able toprovide a lot of value to a user? i actually don\'t think metrics worksthat well for stuff like market size. so you just saw this example that stewart,they thought that the market size for chat was 100 million a year in revenue. it turns out to be 100 times bigger. it\'s just very hard tofigure those things out. i wouldn\'t spend a lot of time on that. i think you want to knowthat whatever you\'re doing, someday there\'s some path forit to eventually get really big.

but, you want to really apply your metricsto how can you differentiate your product, and what\'s going to be possiblethat wasn\'t previously possible. so, okay so next, now you\'re buildingyour product and what should you measure? just in any guesses, if you guys haveproducts, what are you measuring, what would you recommend, yeah? >> [inaudible]>> the amount of users, yeah, yeah, so what exactly would youmean when you say that? >> [inaudible]>> yeah, so how many people use a total andhow often they use it and go ahead.

>> i think [inaudible] people share it but it\'s viral, [inaudible]>> like whether it\'s growing or whether they do something sothat it would grow, yeah? >> total addressable market likewhen you\'re getting started. >> total addressable market, yep. >> user attention. >> yeah, user attention. so these are all great ideas. and, there\'s lots of thingsthat you can measure.

as a startup, you really need to focus. so the core concept i would try to focuson is, users that are getting value today. so, there\'s different indicators of this. you might say,you want to focus on active users. people are going to be active,if they\'re getting value. you might want to focus on revenue. that\'s a sign that peopleare getting value. in the case of a marketplace, where youhave things like buyers and sellers, that\'s usually a case where it wouldmake sense to measure transactions.

any time you have these likedifferent groups, if you say, okay, we\'re going to measure buyers andthen we\'re going to measure sellers then, it\'s actually lot of stuff you can do thatwill make things better for the buyers and worse for the sellers andsome of those things are good. and so measuring transactionsis a way to unify them and align your work on things thatare going to benefit both sides. so, if you have a two-sided market oranything like that then, you want to measure transactions ortransaction values. so things like ebay,

they\'ll measure total value of allthe goods that are sold on ebay. so to start,this is kind of a quick thing. and you go a long wayjust by measuring this. if you\'re not measuring anything, you\'re susceptible to all these cognitivebiases that stewart talked about. there\'s this all these kinds of problems, you\'re just not going toknow what\'s going on. so next was most important thingto think about is retention. so, first, i want to make sureeveryone understands what a cohort is.

so, a cohort is all the users who firstuse your product in a certain time window. so for a startup,you might make the time window a week. so you could say, there\'s a cohort ofall the users who first use your product in the first week that it launched, andyou can track those users over time. there\'s a cohort of all the users thatfirst use the product a week later. and, you measure all these cohorts andyou can track them and this tells you what\'shappening to users over time. and this is a reallyimportant way to balance, you\'re just measuring the total valuethat all the users are getting so

this is a graph of totalof total cohort usage for a sort of relevantly old internet product. so the blue line is users whojoined this product in 2004. and that\'s as you can see, and so this the y axis here is whatpercent of them are still using it. and the x axis is time. so overtime you just follow the blue line. you can see usually overmonth there\'s fewer and fewer people that joined the productin 2004 that are still using it.

and this is usually the same story forall these different cohort lines. so what do you think is going tohappen to this product in ten. this is about,let\'s say this is about six years. the scale of this graph. what do you think is goingto happen six years later? any guesses? >> so kind of the opposite. >> it\'s going to turn around. >> yeah.

>> yeah, it could. i think something would need, something different would haveto happen to change that. >> no one using. >> no one will be using it. yeah, so this is,it\'s hard to know what would happen. i think probably everyone isgoing to stop using their product. i think some companies willmanage to turn these around. but it\'s very hard toobecause you get fewer and

fewer, you get less andless usage from your old users. it\'s also usually the old usersare what caused you to get new users. and soif you have this problem it\'s pretty bad. this is a pretty scary place to be. even if you can have a ton of usersyou can have total usage growing. but if usage from everycohort is declining then you\'re going torun into big problems. so there\'s this idea calledthe ring of fire which is a way to kind of visualize this.

so unless there\'s some kind of lower boundyou\'re going to burn out all your users. and so here\'s the analogy,imagine there\'s a big field of let\'s say of dead grass andsomeone lights a match in the middle. you\'re going to get a fire andthe fire\'s going to grow and grow. but then in the middle of the fire you\'regoing to start to run out of fuel and it\'s going to burn out. and so you get this ring. and the ring is going to get bigger andbigger. and in the center you\'re going toburn out more and more of the fuel.

and so the fire will get bigger andbigger but eventually you\'re going to run out of thewhole field and the fires going to stop. you don\'t want this to happen. here\'s an animation i madeto try to illustrate that. >> [laugh]>> so here\'s what you notice. so it looks great at first. there\'s more fire every framethan the previous frame. but in the end there\'s nothing, right. so you don\'t want this to happen.

and it\'s really important tointernalize this concept. because if you don\'t you can fool yourselfand you can do a lot of things that are going to cause you to get more andmore usage over time. but at the expense ofthe existing users and if you don\'t do this eventuallyyour company is going to die. i think some recent examples are i thinkgroupon was a good example of this. they got a huge tons of usage,all this merchants but it wasn\'t a good experience forthe merchants in the long term. and so they all stopped using it.

and then i think more recentlypokeman go is another example. it got really big, lots of users butit just kind of faded away. and so you don\'t want to build a fad. you want to build somethingthat\'s going to last. and so you need to makesure that you\'re measuring the usage from the existing users. it\'s not just that you\'regetting more users. so you need to make sure the existingusers keep using your product. actually if this can go in reverse you\'rein an extremely powerful position.

there\'s a small number of productswhere the cohort usage actually will increase over time. one example is whatsapp. this is i think most messaging apps, as you get more of your friends on the appthere is more people to send messages to. and so they are able to get thisincrease in cohorts usage overtime. uber on the rider side. this is another good example as peopleget used to uber they use it more. as the pickup times comedown they use it more.

as prices come down they use it more. and souber\'s in a really strong position there. it\'s not true,actually on the driver side. so uber has problems ondriver side because they don\'t have this property there. and then facebook you can seethe numbers they report publicly. they keep on their total usage is growingfaster than their number of users. so you know that they must have thisincrease in usage per user overtime. so if you get to this,this is very hard to get to this.

but if you get to this point thenyou\'re in a really strong position and one way to actually thinkan idea is to whether. as the product gets biggeras you get more users. as you develop the market is itthe kind of product where people would use it more over time. i think that\'s another lens you canapply as you\'re considering ideas. just one more data point, this venturecapitalist named tom tunguz did a study on what predicts valuationsthat startups raise money at. and it\'s just a simple correlation study.

and these are some differentfactors in how much they correlate to the valuationthat a startup\'s going to get. first is growth of revenue,and that\'s 0.18 correlation. second is just how muchrevenue they have in total. that actually carl is a littlebit more strongly than growth. but by far the strongest correlationis account expansion which means people spending more overtime. it\'s the same thing asretention increasing overtime. so andthis correlation is extremely strong.

so for a complex number like whatthe valuation of a company is going to be. the fact that you can explain pointfive four out of one of the variation in that number by the single variable. about whether the company isincreasing revenue per user over time. that\'s a sign, it\'s almost likethere\'s just two classes of companies. one class where there\'sincreasing revenue over time and another class that gets totally differentevaluations where this is not happening. so the main thing i just want you toremember from today is you should measure your retention.

that\'s going to lead you to do a lotof things that will help you in all kinds of ways. it\'s going to help you,it\'s going to help you grow. it\'s going to help you retain your users. it\'s going to help you builda product that users really love. so next i just want to talkabout exponential growth. so retention is the most importantthing but it\'s not sufficient. you need to have a product that\'sgoing to retain your users. but that\'s not necessarily enough to makesure that you\'re going to be able to

build a big product and reach everyone. you also need to make surethat you are going to grow and by far the most powerful way togrow is through exponential growth. this is just a few functions spottedthe red line is linear growth. and blue line is i thinkis cubic growth and then the green line is exponential growth. and the linear growth path thisyou could do these with few things like continually doing pr. you can get more and more users.

eventually, there\'s a limit to howbig you\'re going to be able to get. it gets expensive to run advertising,it get\'s expensive to do these kinds of things that attracta fixed number of users. but if you can do things that cause youto get users in proportion to the number of users you already have, then that\'sgoing to get you to exponential growth. so there are other places to learnabout what to do to grow faster. but as you\'re measuring your growth, youwant to make sure that the way you measure it is aligned with what you want, andif what you want is exponential growth, then what you need to measureis the rate of growth.

so a good way to measure this isjust the percent increase per week in how much usage you have,or how much revenue you have. this is a tweet from paul grahamthat i especially agreed with. he said, a suggestion for really toughfounders is instead of graphing your revenue, graph the percentgrowth per week. i don\'t actually know why he said thatthis is for really tough founders. i actually think this is what\'s going tomake you really tough if you measure this. i think the causalitycan go the other way. so this is a veryimportant thing to measure.

the reason he says youneed to be tough is that, it\'s going to be hard to keepup exponential growth, and it might not feel good to just seethis line that doesn\'t increase. but i mean you can alsomeasure your total revenue. there\'s no problem with doing that, but just make sure that you\'re measuringthe percent growth per week. because a lot of the time, there\'s going to be things you can dothat don\'t cause exponential growth, that would actually help youmaybe more in the short term.

and you want to focus on the stuff that\'sgoing to cause the product to grow, in the short-term and in the long term. which is basically thingslike viral growth, getting users to invite more of theirfriends, getting word of mouth spreading. getting some kind of dynamic wherethe bigger your product gets, the easier it is to attractthe next user to it. so next i want to talk about iteration. so one of the most important things, oneof the most important advantages you have as a start-up, is that you caniterate faster than a big company.

there\'s all kinds of things that causeit to be hard for companies to change direction, and you just don\'t have theseforces when you\'re a small company. and so if you can iterate muchfaster than other companies, then you can go through ideas faster. you can go through changesto your product faster. you can go through changesto improve retention faster. you can go through changesto increase growth faster. it\'s going to add up to a really,really strong position. but at the same time you have to beiterating in the right direction.

and so one of the reasons metrics, evenjust simple metrics, are so important, is that they let you know whetheryou\'re going in the right direction. they let you know whether what you\'redoing is working, and they let you out of these different ways you can gothey kind of help you stay aligned. one other point i want to make,i think this isn\'t mentioned too often, but you can measure howfast you\'re iterating, and you can measure all thesecomponents of iteration. so whether it\'s fully quantified or not,you should have in your head, okay, how long did this cycle take us, and whatcould we do to make the next cycle faster?

at quora, one of the metrics we track is,how long does it take from an engineer hitting commits on their code,to us being able to run the unit test, and then andthen get that code out into production. and we try to keep that in likebetween 10 and 15 minutes, from a change in the code base to itbeing totally out to all the users, because that allows people toiterate just much much faster. and if we didn\'t track that metric, there\'s just a lot of forces thatnaturally would cause it to get slower. i think a lot of companies end up,they maybe start out like that,

then they end up releasing daily, orthey go to weekly, or even monthly. i\'ve heard of teams that are on likeevery three months they do a release. and you just imagine,how could you possibly iterate if your releases are only going monthly,or even weekly? so next, there\'s a spectrum ofhow seriously to take metrics, that i see a lot of companies stumble on. so on the one hand, there\'s nottaking your metrics seriously enough. and this might mean things like,you don\'t have metrics. it might mean that you have metrics,they\'re on some page, but

no one ever looks at that. it might mean that you have metrics andone person in the company looks at it. but no one else looks at it, and the issues that happen onthis end of the spectrum are, you kind of like, you can fool yourself,you go around in circles, you just like lose sense of whatthe right things to do are. and then there\'s anotherend of the spectrum, where you can actually takemetrics too seriously. so you\'ve gotta remember thatmetrics are an abstraction.

there is where you\'re actuallycare about underneath is, let\'s say the value to the users, andmaybe the long-term value to the users. and so a good metric will bestrongly correlated with that, but it won\'t be perfect. and so you don\'t want to forget thatthere\'s a reality under the metric that\'s not exactly the metric itself. and soif you focus too much on the metrics, then you start to get into things like,there\'s these ways to game the metrics. and we can make the metrics go up, but

actually we\'re doing some things thatare going to hurt us long-term, and those things are going tooutweigh the short-term benefits. so i would really make sureto be careful about that. i\'d say, personally, i think the idealis to be somewhere in the middle, and i\'d say you want to be very close totaking the metrics extremely seriously. not too seriously, but i think most companies bias toward notbeing serious enough about their metrics. if you\'re in the right, if you\'re middlein the right spot then you\'re learning, you\'re making good decisions,and you\'re informed.

but you\'re still aware that thereis a product underneath, and there is a strategy and there\'s moreto the world than just these metrics. i see a lot of times companieswill oscillate between these, like they\'ll say,they\'ll get really focused on metrics and then that causes them to do somethings that they think are bad. and they\'re like thesemetrics are terrible, and we\'re going to throw away our metrics,not focus on them at all. and then you get into this world where youdon\'t actually know what you\'re doing, and there\'s no accountability.

and so oscillation is not very productive, you want to just make sure you findthe right point and try to stick there. so finally, i want to just talk about someof these psychological aspects that come with metrics. i think one of the big things thatcome when you start measuring things, is you just get closer tobeing in touch with reality. and that can be painful, because sometimes the metrics are going tobe worse than you thought they were. so sometimes they can be better thanyou thought, and that\'s great, and

everyone feels great. but a lot of times if you weren\'tmeasuring something chances are you weren\'t even aware of it, it washard for you to make it good, and so it won\'t be good. and sothat means that as you set up metrics, there\'s going to be some pain,and that can cause problems for companies, i even think at the leadershiplevel, this can get even worse. actually, i think the majority of leadersout there in the world rely on some amount of delusion they\'recreating among their followers.

to get them excited, to have them workingon this project in the first place, and metrics can get in the way of that. and so people get into this, then you getinto this question of like, okay, well, what should you do as a leader? and this is where i come out,after thinking about this for a long time. i think you want to be learning from thepast, so you want everyone fully aware of what has happened, what has worked,what has not worked. you want to be totally in touch withthe reality of where you are today. so if things are good,you should know they\'re good.

if things are not good, you should knowthey\'re not good, you should know exactly where there are problems, andwhere there are not problems. but then, those things are not in tensionwith being optimistic about the future, you should be optimistic about the future. because there\'s so much potential, and the reason you\'re doing a startupis because of this potential. and if you don\'t feelconfident that the path you\'re on is able to get you to a placethat you feel optimistic about, then that\'s a good sign that youmight want to change the path.

and you should do something about that,because it\'s totally in your control. so there\'s not really a tensionbetween acknowledging that things are not in a great spot, and having peoplenot be optimistic about the future. so as long as you just make sure thatthe reality is that, and i don\'t think you should make people optimistic about thefuture in a way that you don\'t believe. i think you should do the things you needto do, to get yourself to be confident, and optimistic about the future. and then it\'s easy to getpeople to follow through that. so that\'s all i have, andhappy to take questions.

>> [inaudible] the questionwith the it\'s desk >> [laugh] >> thank you very much. >> [applause]>> all right, see you on thursday.

web design software quora New Zea – WordPress Theme to Display Your Photos