Libertopia 2012 – Teresa Warmke – Obsolete the FDA and Make Your Own Healthcare Product Decisions

welcome to my talk I’m Theresa warm p I’m here to talk about how to make your own decisions about health care products you know i’m all for obsolete in the fda but you know it would probably take a while and one of the most important factors in that is to help people really think about health care products and not just trust that if it’s something their doctor recommends because it’s approved that means it’s good and so my talk is really to explain some of the ways that you can think critically about data you see and advertisements you see about care products i’m also with free aid just like stephanie and thanks for thanks for your kind words stephanie it is a pleasure to work with free aid we’re an organization that has you know that networks with a lot of medically skilled Liberty lovers and we’ve got about fifty members of our facebook group now and I’m delighted a chip is one of them here and and Stephanie and several others and we’re here providing first aid services as well for the festival so far we haven’t had anything too serious I know and and so that’s great and we are I am also going to be doing CPR workshops at one o’clock today and then one o’clock on tomorrow as well and it’s also very refreshing to see how many people were interested in mutual aid for Stephanies talk earlier and CPR is a good example of mutual aid people helping each other without having to you know being empowered to help each other without waiting for government providers so all right so some of the things to look for in a study to summarize is the study design appropriate for the question being asked is this study well conducted where subjects informed of risks beforehand it’s very important ethically ethically how strong are the results how generalizable are the study results beyond the study population and what percentage of patients benefit from the treatment so these are all important questions to ask and just a poll you know how many people here have either had a procedure or prescription drug or a loved one that has had a procedure or a prescription drug you know how many people here have have really looked into some data about their products good good excellent I always do I know too much I work in the I work in the medical device industry and I just know too much about what FDA doesn’t doesn’t ask and and how data can be interpreted differently in different circumstances so I’m here to help help provide information about how to ask good questions about the products that you or your loved ones receive so there’s two primary problems with health care product data one is confounding so what that means if if if choosing the treatment or exposure is related to other causes of the outcome you might see a spurious relationship so I’ll talk a little bit more about this in upcoming slides and bias is that if the assessment of the exposure and outcome are related in some way you might also see a spurious relationship so I’m going to talk about each of these ensure so confounding an example of a confounding error is you’re looking at the relationship between coffee consumption and cardiovascular disease if people who consume coffee are more likely to have unhealthy lifestyles maybe they smoke maybe they don’t exercise very much then you might end up not making the right conclusion unless you control for that those confounding variables and then another example is it can work the other way as well if you’re looking to see if taking a multivitamin supplement will help some kinds of cancers and people who take vitamins also have healthier lifestyles you might see a confounding relationship that isn’t causal you might make a conclusion erroneously that taking vitamin supplements helps prevent cancer but it might be related to all these other variables that are going on in that study the other type of problem is bias and that occurs when there is Miss measurement of some kind and when there is poor measurement in both the exposed and unexposed populations then that can dilute a true effect so that you wouldn’t wouldn’t see a effect that really was there in some cases and in other cases when there is different measurement of one outcome versus another tip to the exposed population in

the unexposed population then it can make it seem like there is an association when it’s not really there an example of different types of biases recall bias so this is kind of well understood people with the outcome may be more likely to over remember those with Althea outcome are more likely to forget so it’s a you know the example we’re giving here a cellphone use if someone develops a brain tumor and and is asked you know about cell phone use they might remember oh yeah I used cell phone but if they don’t develop a brain tumor or they’re not likely to remember that at all ascertain meant bias occurs when assessment of the outcome is different between the exposed and unexposed so for example hormonal contraception and embolism women who use 12 pills may be more likely to be screened for embolisms because a more frequent contact with doctors and concerns about the association and the more you strain for it the more you’re likely to find it so that’s an ascertain meant bias and then another type of bias that happens a lot in health care product studies is lost to follow-up so if you’re doing a study over a period of time the patients who you know usually there are predefined intervals at which they’re going to be measured for different outcomes and if what happens is that some of the patients drop out they don’t want to continue the study or something else happens and that’s not always random the way that happens you know so if the drug causes serious side effects more of those taking the drug may drop out than those who don’t take the drug and so then the results might look better than they are and yeah so that may look like there are lower risk of adverse events and group that dropped out so here are some ways to control for these problems one of the best ways is randomized control trials not all health care products go through randomized control trials but the it really helps eliminate some of those issues randomization from a treatment group and a control group really helps control the confounding variables if they’re really randomly assigned across the population you know to ensure that the comparison groups are similar on all characteristics other than the measurement other than the thing you’re measuring so that’s important with randomization and some randomization schemes are better than others so you know if you look at a study be sure to think about that and then the control group it makes you making sure there is a control group has to be adequate and for drug and device trials if there is no no an effective treatment new treatments are tested versus placebo but if there is if there are other treatments for the condition right now like for instance blood pressure or something like that there are currently effective treatments for that you have to treat it you have to test it versus the standard of care and it’s tricky because sometimes the standard of care can vary by location and in different countries and you name it a blind is another method to control for some of these issues I to prevent bias and if patients know they received an active drug they may be more likely to report improvement it’s kind of known as the placebo effect you know they’re more likely to report improvement and they’re also more likely to report in effects if they think they have the active treatment but if patients know they received a placebo they may drop out of the study and they may decide to take the drug on their own and it can cause issues and if a researcher or evaluator knows that the patient received the active drug they may be more inclined to see improvement and but one of the issues you can’t always blind subjects or researchers if you’re if it’s a really invasive procedure some kind of surgical procedure or something it’s really not ethical to do a sham and not really do it you know so it can be really difficult but but blinded studies are generally more free from bias than others some of the limitations of randomized control trials the study population is usually very specific so you’re trying to isolate the variable in question and so generally they they get you know from silikal companies and medical device companies can try when they do randomized control trials to be have a very specific population that might not always be applicable to the broad population you know this happened with vioxx for example they studied a narrow population and then it got used for a lot of different things that weren’t always well studied randomized controlled trial should only be

conducted when there is a state of equipoise there’s a legitimate question as to whether the treatment is beneficial or not if you know there has to be equipoise between the treatment group the control group to make it ethical even you can’t conduct a trial where drug with known effectiveness is withheld like when I was talking about blood pressure earlier it wouldn’t be ethical that’s it tell people you have to stop taking all your meds so that we can study this new one that’s not appropriate either if it really is standard of care although some blood pressure medicines are better than others let me put that caveat out there and you can’t conduct a trial that’s where exposure is likely to be harmful to a subject so you can’t assign someone to start smoking cigarettes to study the effect of it and another example of this is like the verb later that we have that free aid I worked at the compass that made it when we designed it we made the waveform extremely efficient so that it could be in a small device that like that one and it was so not according to the standard for different blade or wave forms and we were able to do a clinical trial at that time because back then when people were receiving implantable defibrillators it was standard of care to put them into ventricular fibrillation to test the implant the device but the implanted device is so good right now that that they no longer need to do that so no one ever gets put into VF anymore which is a good thing we got is not a good thing Ian even when you have a defibrillator right there that you know is going to bring them out of it but because of this it’s not possible to innovate new defibrillation waveforms anymore because in the case of an automated external defibrillator for cardiac arrest cardiac arrest doesn’t occur often enough to study it would be impractical to conduct a study of aeds and then there are also informed consent challenges the victim is not responsive and so in its moments count so it’s really difficult to do research into defibrillators and so so right now it’s very difficult to study this and it’s not a problem I have all the solutions to it’s something I’m very interested in talking about how do you how do you handle the situation when you need to get informed consent for experimental therapy when the patient is unconscious and their loved ones and whatever are not likely around or can really understand in the short time that you need to go ahead and try to innovate so it’s a real challenge that’s not going to be solved by getting rid of erm anout of the picture they they struggle with it too but I’m glad I’m glad we were able to study the way from that centaur defibrillator it would just be nice to be able to innovate further in the future so when when a randomized controlled trial for these various reasons aren’t possible observational studies are generally used and there what happens is that the treatment is not allocated to subjects by researcher but chosen by individuals so they they decide if they want to have the treatment or not and then so the limitations of these is bias and confounding because it’s you don’t have the randomization that sells for the confounding and you don’t have the control group it’s just just people you’re just looking people that are taking the product but one of the things I didn’t put on here but one of the things with observational studies that can be helpful is if you can see that they predefined what they expect to see in the population that took it so when you when our researcher pre defines the outcome they and the hypothesis that they expect to see then that really helps minimize some of these things but not completely there are different types of observational studies a cohort study is where researchers select a population or cohort and they assess their level of exposure and then follow them over a long period of time and and to see if they develop the outcome there are others where it’s a case-control where they select a group of people with an outcome and a group in a group without sometimes with a historical control and and then compared the levels of previous exposure in both groups if you’re using a historical control a good question to ask is what was the standard of care at the time that hit that historical control population was exposed to the product before and because it can change over time quite a lot practice of medicine and all kinds of things can change and or if it’s in another country then the you know the better that the better that you control the better that you match your controls the better off

you can be ecological observational studies are where researchers select populations not individuals and my my colleague Mariko Geiger helped me put these studies together and she’s an epidemiologist and I had never heard of this before but she knows all kinds of interesting things and so an example of this is the effect of bicycle helmet laws on the rate of head injury and these studies can be pretty severely flawed because they’re looking at groups of people and there’s a lot of confounding stuff that goes on with it and you know people want to see that bike helmets help and so it has all kinds of biases to some now moving on to interpretation so when you’re looking at an advertisement or you’re looking at study or you know and I feel like me and you see an advertisement after a doctor recommend something and you look it up on google and you look for scientific papers you’ll see different things reported in those papers and statistical testing you know I took a class in statistics in college and I just kind of it was really difficult to understand how to apply it but after working in the business for so long I’ve gotten I’ve gotten to know a few things working with my statisticians that I work with so testing statistically is based on the possibility that sometimes associations are simply due to chance and so if a statistically significant relationship is found it’s unlikely that the relationship is by chance but it is still possible there’s always a chance that that you saw a good result that was just based on the fact that there weren’t quite enough people to be sure that that was right when a study is looking at many different comparisons the possibility of chance error is multiplied so so when you see that reports decrease in deaths decrease in myocardial infarctions decrease in lots of different things that they’ve looked at in the same population the chances that they’re all good is multiple you know the chance if they report all good results the chances that there’s an error and at least one of them is multiplied when a study is looking let’s see here oh sorry replication is really critical to scientific inquiry the more times the result has been replicated in separate studies by separate researchers the more you can trust it in an example one of the one of the examples I research before this talk was medical marijuana and I was very surprised to see how few studies there are on it um yeah IIIi don’t know I just sort of expected to see more just based on all the policy talks that go on and all the other stuff but any problems Christy yeah she’s asking can they study can they study medical marijuana what I heard was that when I heard one hour when I saw when I researched this is that there is one I guess a lab I don’t know 11 site in the US where you can requisition marijuana for studies and and the studies I thought I only saw two studies and they were in very narrow populations and very limited numbers of subjects a single site I think where they tested marijuana cigarettes versus non marijuana cigarettes and they called it blinded and I find this impossible to believe so I am very suspicious of those studies but I would love to see more and so you know I wanted to see more ok so maybe ribs yeah there’s plenty of information of that ok so maybe that’s what they did but I looked for that in the paper and I didn’t see it so I was suspicious but it would have been better for them to explain you could still you know that it really was blinded but anyway replication is really important I would love to see more not want to study is replicated and maybe not you know just in the US that’s really locked down but I’ve heard other countries like the Netherlands oh yeah there you go yeah do you more do more studies um that’s that’s really helpful and sometimes when you look at medical product studies you see something called a p-value and if it’s less than five percent which is reported as p less than 0.05 the chance error it’s considered acceptable statistically significant this is somewhat arbitrary you know so I want people to know they’re saying they’re saying there’s a five percent chance this might this result might be wrong and so just keep that in mind when you see things like that or sometimes you see p equals 0 point 0 0 or 1 you know quite a few zeros before the one you can have a lot of result is probably right but if it’s

like point 0 for 9 you know you’re kind of you know they usually would say less than 5 in that case but you know that’s that’s sort of a ballpark and it’s not like there’s it’s written into our regulation but FDA never approves something as far as I’ve seen that’s more than point 05 sure yeah Johnny those my feet reported that way but that doesn’t make up for a bad designs Valley design aesthetic very good point Jody gently saying that even if you have a P equals less than point 05 that doesn’t necessarily mean the results of something you can count on because and that also has to do with statistical significance doesn’t always translate to clinical significance that’s my other point on here that just because mathematically the result would turn out that way because maybe the way they designed it or whatever to do that yeah confounding and bias yes that happens a lot actually and so that’s a really good point to think about what that outcome does that outcome really matter to you if you’re able to show is to two significant result in terms of mathematically in terms of reducing blood pressure from you know blood pressure from like 150 to 149 you know so what you know it’s you know that’s not very much it doesn’t really matter and another example is so an example of sometimes the way you see this patients who receive product a had a significant reduction in cardiac death and myocardial infarction compared to patients who receive product B and then you’ll see it in parentheses p equals 0.003 I took this off of study I looked at but you know a question on that one I would ask is are there would they adjust for multiplicity because they’re reporting both death and myocardial infarction so I would ask questions like that a lot of the time they don’t I would even say most of the time they don’t so be aware that even with a significant result like that if they report multiple things it might be less than that really and then my other prerogative question is would you believe a conclusion that was p equals 0.05 one this wouldn’t get approved by the FDA but it’s probably okay you know and so I just want to get people thinking that way oh yeah that’s a really good point yeah it’s depending on depending on what the condition is you can really you can have a different level that you’ll accept a level of scientific rigor and like what I was talking about with defibrillators you know thank goodness we have them but I but it’s really hard to study them and thank you and I well I will I will I will use it as if you do so you could have that yeah there are there are some people would do not resuscitate bands I’ve even heard of tattoos you know all kinds of stuff and I absolutely respect that but the odds difficult yeah Sharon there are two more caveat about studies a recent issue of scientific american mind has an article about professional subjects and the pit the researchers are so here to get subjects they’re not as fussy as they should be yeah who gets to be a subject that’s a really good point too yeah Sharon saying that some researchers are so eager to get study subjects because they get not only money for each patient that they include in a study from the manufacturer of whatever product is under investigation but they also in probably an even more important motivator to think about is that they research they get to be published and and that really helps them advance in their careers and just be careful when you’re working with investigators there that can be a bias in selecting the patience for the group and and sometimes you know sometimes papers put it in the fine print you know that you know some of the subjects didn’t have all of the inclusion criteria but we and we presented it and sometimes they’ll present the results without those patients and sometimes I’ll present them with the patients and just be careful when you’re looking at that it’s it’s it’s definitely an issue so yes questions by the actual Oh absolutely and that’s a really good

segue into my next book here is is the comment here that it’s really important to follow the money it’s always important to follow money if you’re seeing a really something that seems to be true or something that doesn’t make sense to you or the numbers just aren’t adding up to you that or even if they are you know just be cognizant of where people acting the funding you know the funding for virtually all medical product testing comes from the companies that make the products and that’s the only way you can get the research done but you know just be aware of you know they and they pay for they pay for health care also for the subject so like routine follow ups and stuff and so sometimes people can get free health care if they sign up for a study and that’s to sharon’s point earlier that you can get professional subjects that way also so there’s incentives for them that way and you know that’s good oh yeah question is that place between the need for all these drugs and the production of food and it’s like what was was being passed off the spoon and like it spins micronized why we need these drugs cure the diseases there could cause for things were eating uh yeah I’m not I’m not the question is about foods and and comments on all these drugs that are being used to address issues that are caused by foods and and things like that i’m actually not super familiar with a lot of the food food issues i know i know a little bit about you know wheat allergies and things like that i do see that because you know medicine has really changed over the last century even and to where doctors are spending less and less time with patients and if there is a drug that is purported to be treating something that’s what they go to first with that because it’s hard to get a patient to change their lifestyle it’s hard to get someone to quit smoking it’s hard to all these things are hard and they take time and time that you know physicians don’t have and and the other is it’s not all their fault sometimes they can get sued if they don’t prescribe and because it’s because that because standard of care because it’s so easy to write a prescription and and it’s so hard to do these other things that then it becomes standard of care and then they can get sued if they don’t follow standard of care so it’s it’s a it’s an issue for sure but it’s something that the more people that are aware of that being an issue the better empowered we can be to help make good decisions and and work on things and help each other work on things rather than always relying on the prescription pad of the doctor and you know some of them are great but I had a doctor that was fantastic and always took time to do the harder things and he got out of it to be in practice anymore thought about that that and he was an ob/gyn and you know the insurance for those guys is crazy so he couldn’t make money with the time he wanted to spend with patients so it was sad anyway I’m going to go on just have a little bit of one more topic to cover here one of the things that FDA never makes people report in the labeling or anywhere else is responder rate and this is really important and it’s it’s also important to those of us who care about being individuals versus collectivists you know and especially when it comes to cancer medication chemotherapy so there is a there is a organization called the ASCO that looks at different chemotherapy drugs and it’s it’s really all over the place there is no one chemotherapy there are different combinations of drugs for different cancers and have been studied in various different cancers and this is an example of something that the ASCO came up with as a tool for doctors to talk to patients with that I really like to see but I never see it and drug product labeling or anything like that this is what you’re looking at here is sorry it’s kind of hard to see with the blue background I didn’t didn’t think about that but what you see here are the pants that would would die anyway with or without the drug the red is the patients that survive anyway with or without the drug with no effect in this 13 over here is the patients that benefited from the drug and and the CEO was recommending and this one is is worse it has only ten percent benefiting from the drug these are chemotherapy drugs these are nasty drugs and if you knew that only 10 people benefit tempers I’m sorry ten percent of people benefit from it would you really want to do it I think that’s up to each person to decide you know I think some people would say yes I’m willing to go for it they live two months longer you know this ten percent of people live two months longer with us

taking this nasty drug I’m going to go for it and try to try to do it and try to make it but like and especially when it’s chemotherapy if it’s only two months and only ten percent my goodness i think i would probably decide not to do it but this and that’s exactly what was going on in this study and this is something i never see i never seen labeling i’m glad the ASCO is recommending it for tools for communication with patients but I just I just don’t see it or happen and so so one of the things I said um this was actually this quote and I read it to you this quote came from a recommendation saying Oh chemotherapy drugs work after all but it’s really telling that even in this article that was recommended in saying how it’s proven effective they what the issues are with it that I have with it well it’s true that chemotherapy decreases a woman’s risk of dying from her breast cancer the vast majority of women do not individually benefit from chemotherapy in in order to save that one third and this in this case it was one-third not a tenth in order to save that one third we have to treat most women who segregate into three groups those who would have done well without chemotherapy who are treated unnecessarily those who would do poorly regardless of chemotherapy who are also treated unnecessarily and those for whom chemotherapy is the difference between life and death and so these standards of care and public policies I’ll recommend that everyone get it but you know I don’t think everyone would want it if they really knew all the facts and can send the other thing to consider is this conflict of interest follow the money because chemotherapy has to be given over a long period of time a lot of cancer you know oncologists set up businesses that would administer the drug so instead of just writing a prescription which doesn’t have as much conflict of interest they would buy the drugs in bulk from the drug companies and and some of them were allowed to have more profit than others by the government so um and just think about the bias in that into the conflict of interest that can crop up from that you know these these doctors that are recommending a certain drug to a patient nasty drug to a patient and and there were studies that showed that there were differences in I can’t remember the results and i didn’t write it down on here but there are some that there were more prescribed that had the better profit allowances than the others and that’s just you know unfortunate and i think i would like to believe that most doctors wouldn’t knowingly do this but definitely follow the money always it’s it’s really important to know and think about who benefits from things and do some research online before you have to do something interventional like that yeah Stephanie oh for sure yeah I have seen a big changes in the industry since I’ve been where I’ve been working in it for 20 years and you know they’re there have been a lot of voluntary agreements among industry associations to get rid of a lot of the free giveaways that’s definitely saying can cause physicians to have biases but of course as soon as the industry it’s not to do it anymore then the government makes it a requirement that’s kind of the way it works and then they claim that that it was all their idea you know and that that’s generally what happened so but and so there are biases created from the giveaways and things like that but I don’t want it you know i want to emphasize again that probably the publication bias is even greater if if these investigators and physicians know that they’re going to be able to be a principal investigator and be named on the paper that results from the study that is very motivated to a lot of them and so be cognizant of that too some of them are just really interested in advancing their careers getting more patients expanding their business that’s how they make profit to you know a lot of them and and that’s comes a lot from publishing and that’s what they know they the most of the studies that are going on are funded by health care product companies and so they they would obviously have a bias toward those ones that pick them as the investigator most often so to conclude I just want to reprice what I said at the beginning you’re evaluating a health care product for yourself or a loved one ask is the study design appropriate you know if the

observational study did it have to be why didn’t they were randomized control trial a lot of a lot of observational studies are in populations that could well have been a randomized controlled study some of the Mart though so think about that you know are they studying the right outcomes is another part of design you know what we’ve talked about earlier is it well conducted you know did they did they were they meticulous in their collection or did they were they missing a whole bunch of people like the one your time point and just report on the ones they had you know like that did they if it’s something like if it’s something like a coronary stent were they really looking for the right things down the road really making the right measurements that you can see what’s going on we’re subjects informed of risks ahead of time that’s just something that’s you know rock-bottom critical informed consent you know some there was a big flap about a decade ago with some cancer studies that did not adequately inform patients of risks before they did this interventions and that’s just unacceptable how strong are the results you know assuming the result that you care about you know do the report statistical significance and to what extent how generalizable are the results is is the you really can’t believe that is something in patients with a certain type of severe severe treatment resistant high blood pressure for example would that be applicable of someone with blood pressure of 140 / 81 or something like that you know how how do you how much do you believe that the study population is applicable and then what percentage of patients benefit from it like those chemotherapy drugs you know what I really think critically about that because that should factor into any decision and whether you take what recommend it to someone you care about so yeah I think that’s it yep yeah Chrissy oh I generally try to go the healthy route first now find the natural or whatever don’t like drugs yeah me too but in the case that i do i’m probably not going to go to all of this have you thought of like creating a third party rating system absolutely because i would told you know that would be a place together i would love to do that with aid and i know so much about what’s available you know with fda publishes a lot of stuff and a lot of it is good information but some of it is missing and some of it is biased because once they r approved something they want to defend it no matter what because they don’t want to ever mistake exactly and so um so there are issues like and free aid never accepts government money we never will all of our funding has been very grassroots and we really appreciate the generous support of our donors but I it’s a project that i’ve listed that i would like to do with free aid in the future I’m busy with actually happy she’s she’s saying you know you definitely contact us via free a calm we have a contact us form on there and I don’t maybe we do have our phone number we do have a phone number that forwards to us as well it mostly goes to voicemail but that’s another way to contact us if you have a specific question and want my advice I’d be happy to help and mariko Geiger my colleague who works at Harvard and the epidemiologist who helped me with this talk couldn’t be here but she you know she and i would be happy to help with questions that come in for us yeah Sharon will hear another creepy caveat what are used to work for one of the companies that oversees drug testing and he quit he said you know I didn’t want to take prescription drug again after he saw Jackie panky that went on oh for sure and I don’t know how we learn about that that’s a tough one but just be aware that not all of these study results are on the up and up yes I totally agree Sharon saying that her partner stopped taking risk you know even prescription drugs just after seeing all the stuff that goes on after working with it and an i do too i I avoid drugs as much as possible and even some foods drugs you know again you know like I avoid drugs as much as possible whether the legal or illegal i don’t care if they’re legal or illegal i don’t i don’t make that distinction like some people do and it’s something i think critically about no matter what it is and yeah I’ve seen too much to it and drugs have systemic effects on the whole

body that you can’t always predict and aren’t always measured so it I’m a little bit more inclined to get a device if you need one then a drug just because I can wrap my head around what happens with the device a little bit better depending on the nature that if I some some device for the nervous system are kind of similar to drugs in the way you don’t really know what happens that’s right yes yeah it could be the question is because the the threshold for prescribing medications based on blood pressure has gone down so now it’s 140 over 80 is considered the point at which it gets treated although it can be lower like 135 over something I can’t remember i think it’s 135 for patients with diabetes also with high blood pressure but and she’s asking if if that could be the result of pushing from drug companies absolutely absolutely it can but look into those results look into not only but this is a good example of a surrogate measure blood pressure if necessarily in and of itself bad you know what it does is that there’s there’s been an association between high blood pressure and cardiovascular events and Sprint’s stroke and you know pretty severe cardiovascular events so the question really to ask there is have there been any studies really done that show that you know 140 a blood pressure of 140 really causes a lot of cardio down the road and there were very few actually as far as I know there aren’t as many as there were in the higher blood pressures and people just get used to it is a very good coral it you know having high blood pressure but I just can’t wait to push a lot of good ones bite but when they when there are vested interests writing a prescription including publication as well as money then that’s before and potentially get a second thing someone that’s less conflicted so yeah yeah yeah Jody’s asking about how readable are the articles some of them are much better than others uh I I guess after just having a lot of experience reading a lot of them I’ve gotten reasonably good at it but some of them are written by people with English’s and their first language and some of them are very heavy in the technical details and jargon and they can be very difficult I agree it’s it’s better when companies can translate the results accurately but you always have to be careful when they do that they’re there are I would like to give some examples in future talks of advertisements versus the studies and and really show people how they can get warped in trying to make it readable and then the other companies do much better yes yeah TV advertisements yeah it can be but yeah those TV ads are so weird you know they yeah they have all those side effects at the end and yeah those are it’s hard to believe they’d be effective for anyone they don’t work for me but but yeah I guess they would be yeah maybe maybe yeah yeah I think yeah exactly I would say not necessarily that’s like for instance you know medical marijuana is natural as they come but there are very few studies like I said so you know follow the data look at the data yes some of these naturals and as much as sure yeah we can get some you know pesticide from China right right so they’re not always as pure and the other thing about them is that aren’t usually as much incentive for companies to trick to test them like there isn’t really because it’s for instance marijuana is easy to grow and so it’s not patentable so who has the interest and studying it for effectiveness and you know thinking it through it isn’t that surprising that there aren’t that many studies you know

there are there really incentives for people or organizations with deep pockets study natural things that can’t be patented so this is another way am I in that intellectual property kind of hurts innovation and science so anyway yeah you had a question but I’m thinking that there’s also a lot of people oh I completely agree yeah his comment is some some people don’t go to doctors enough and others go to them too much there it’s well known that certain categories of people I’ll say you know avoid doctors more than others and are really underserved not only poor people but for instance black people tend to it’s been studied go to doctors less and a lot of it is just mistrust after some of the stuff that happened with Tuskegee experiments without consent and things like that I there are people that are very suspicious of medicine and but on the other hand you’re right there are other people that you know either one attention or feel like they want to have someone care about them that maybe go more often than they need to so yeah good but it also seems to me that you can also saw dark side excessive about ya want everything to be so perfect watch yourself exactly and that’s when the prescription pad comes out and it’s it’s it’s the pressure on doctors to give patients what they want yeah I think we need to wrap up or do I have or what’s the scoop okay maybe one question yes yeah yeah and sometimes sometimes they’re important to follow through on and sometimes they’re not you know so but there are a lot of pressures like we’ve been to get the prescription pad out follow you know follow through on testing or whatever and but there are sometimes pressures for them not to do it cost containment pressures that you also have to be careful about so what are they not pushing on you and sometimes referring to other physicians that you might need they have an incentive not to do it sometimes too so be careful both ways yeah well you just I just want people to know that you can be empowered there’s so much information available now and we’re here to help to you like like Jodi suggested with specific questions until we get our you know third-party database of of information out there you know please you know follow through and know that you can ask those questions we think in cases research and I’m up on this I can yeah oh yeah good idea sure yeah that’s a good idea Jody yes suggesting that we put case studies up there what about that would enable me because I have a day job as well as doing various types of activism so you know maybe we’ll do it a little at a time baby steps yeah that’s a good idea all right well thanks everyone don’t forget CPR workshop at one I sorry daddy drinks so thanks huge a great you