Best Practices in the Public Reporting and Visualization of IDEA Data

>> Good afternoon, and welcome to the IDEA Data Center’s webinar on “Best Practices in the Public Reporting and Visualization of IDEA Data.” My name is Dan Mello, and I will be kicking us off by taking you through some quick webinar logistics and covering our agenda and our attended outcomes for today’s webinar as well For logistics, I wanted to let you know that we are recording the webinar We’ll be muting all participants so that we can have clarity in the line We have a very full agenda from IDC presenters as well as state presenters today, so for the most part, we’ll ask that you type your questions into the chat box, as we also have quite a few people joining us virtually, and this is officially your first reminder that at the very end of our presentation today, we’ll have an evaluation for you to participate in We would really appreciate if you’d give us the feedback that we need to ensure high-quality presentations in the future I did mention that we would be record today’s webinar, and I wanted to let you know that we’ll be posting both the slides and a link to the virtual session on The content is really well organized by topic, and you can also search by keyword Excuse me I wanted to let you know about our presenters today I am Dan Mello, the technical assistance provider from IDC I’m joined by my colleague, Fred Edora We’re going to be covering some of the principles of best practices in public reporting today, and we are also joined by Kara Waldron, Matt Loesch and Ashley Rector from the Ohio Department of Education and Nicholas Armit, Carl Jones and Julie Trevino from Michigan Today, we are going to be covering, as I said, some considerations in design and communication for publicly reported IDEA data We’re going to briefly touch on some resources that exist from IDC to support you in your public reporting, both satisfying the requirements and going beyond, and then we’re going to hear from both Ohio and Michigan about their experiences with their products and their processes that have gotten them to where they are today, and here is your second reminder of the all-important evaluation questions at the very end of our agenda Today, our intended outcomes are to increase understanding about those two principles that we’ll touch on, design and communication We hope to increase knowledge about the available IDC resources that will in particular support [INAUDIBLE] requirements and also going beyond, in particular data visualization and best practices for public reporting And we’ll also hope to achieve increased understanding of how these two states publicly report and communicate their IDEA and special education data to stakeholders, both about the products and the successes and challenges in their process So just basically, why is public reporting important? Of course, it’s required by law It ensures transparency and accountability, and it provides information to parents, advocacy groups and stakeholders Just for a simple mental exercise, we could just imagine what a world without public reporting would be, and we can understand that there is a great spectrum from meeting the basic requirements to really going beyond, increasing visibility, utility of your publicly reported information It does require effort to publicly report, which is required, and so we are talking about some of the things today to help you make the most of that effort by going beyond Briefly, we will be covering today the requirements to report IDEA Section 618 data, which we’re familiar with, including that which is listed on the screen: exiting, discipline, dispute resolution, et cetera here Those are the 618 data We’ll be also focusing on the IDEA Section 616 reporting requirements for public report

This is the state SPP/APR and also the LEA performance on each of the targets set So once you are able to access the slides here, these links will bring you to separate IDC resources on each of those topics Fred and I are going to touch on two principles of design and communication to help you going beyond the requirements, and before I do, let me grab that slide When we talk about design, obviously, there are many questions that go into the way that your data will look and feel and how useful it will be One big decision that states face early on is that the public report of local performance requires that states report on LEA’s performance for each of the indicators, and there’s flexibility in the way that you can do it, and if you report the data by indicator, that is people access an indicator and then see each LEA report on that indicator, that’s quite different from providing something like a profile of an LEA which includes all of that LEA’s information on each indicator We are seeing developments in the field where states are taking one approach or the other, and you can imagine that each is really working for one audience or purpose or another, so we’re going to have a poll question on that in just a moment, but the other thing that we’d like to focus on today is be it design considerations are related to data visualization IDC also has a number of tools that can support you in this work, but when we think about data visualization, the chart type can create The chart type that you choose has such an impact on the way that your data are interpreted The message that you’re trying to deliver is directly connected to whether you’re doing a time sequence, whether you’re comparing two groups, the use of color, the clarity and your choice of font and format, including the way that you the title includes legends and labels, so all of these are quite important, and we have a number of tools and services at IDC, but also, we also see that states are making great choices in the field in order to clarify their data by using data visualization We’ve come now to an opportunity to engage you in two poll questions so Q1 and Q2 I’m going to ask that we have the first poll question: Do you publish your 616 LEA performance by indicator or by LEA? And if you’d also like to respond in the chat box, if you’d like to elaborate, we are curious, if you’re a state that’s considering a change in the way that you do from either by indicator or by LEA, and we’ll just give a few moments for you to reflect on that So we’ve had our first poll come to an end, I think Sophia, jump in if I’ve not ended it correctly, but I’d like to move on Feel free to continue typing in the chat box, but I’m also wondering about whether your state is using currently data visualizations and that same reporting in your local performance report Are you engaging in data visualization? And as you provide a response there that we’ll discuss in a moment, if you want to also reflect in the chat box about any APR reporting data visualization challenges that you are experiencing So question one, do you publish your LEA performance by indicator or by LEA? Thank you for responding It looks like 42 people responded,

and I’ve got no answer a few of those are no answer, so overall, of total participants, 11 percent indicated that they report by indicator, and 27 percent responded that they report by LEA It seems like a little more than double are reporting by LEA Are you using data visualization in your APR reporting? And of all participants in today’s webinar, 22 percent say that they are not using data visualization 14 percent said that they are Okay, so thank you for responding, and if you’d like to respond to those open response questions at any time, you’re welcome to, and at this time, I’m going to turn it over to my colleague, Fred Edora >> Thank you >> I’m going to pass that There you go. Thanks, Fred >> Yeah. Thanks, Dan Thank you, and that information is very helpful as you have taken the poll Feel free to, as Dan mentioned, continue answering those questions in the chat if you’d like to do so So another part of communicating information related to public reports is all about the actual communication process itself, so as Dan spoke about the design of your reports and several very critical important elements related to the actual report itself and whatever software you use to design those, there’s also the communication process as to how you send those reports, how people view them, and we’ve kind of At IDC, we’ve put this in four critical buckets here in terms of how What you need to consider in order to communicate data effectively It’s very central to this, and as you hear from the states here in just a couple minutes, I encourage you to think about how some of these play into the reports that they build, so the first element here is audience, and who is viewing the public reports, or who’s viewing your state’s public reports? Depending on the audience really may change significantly how your report is built, how it’s disseminated, what kind of message is being presented, that kind of thing which also goes into message here, the second element Other than meeting requirements, what will best serve their needs, and who’s specifically looking at those reports, and in terms of the message, what will serve their needs in order to understand the data that is being presented to them? Dissemination, how do they best access those reports? Depending on the audience that you have, whether it’s stakeholders or researchers or other state staff or just the public in general, those four audiences that I kind of explained there will definitely be a different vehicle in terms of dissemination and how they can best access those reports whether that’s what kind of web interface, what kind of software do you use, that kind of thing, and finally, accessibility, how do you ensure data are accessible? And when we say accessibility in terms of communication of reports, what we’re referring to here is, is the stakeholders or audience that are viewing your public reports, are they understanding the data that is in there? Are they understanding the formulas that are being presented or that are being used or calculated within those reports so that they can really, truly understand what is going on in terms of the report? Now, design certainly has some part of that, but also just explaining some of the formulas because as we all know, if you’ve been doing any kind of public reporting with IDEA data, some of the formulas can get complicated, and how are you explaining those definitions and some of those formulas to your stakeholders? Other communication considerations that would be important are things like data integration: What are the benefits, challenges and opportunities related to that? Is there special education data integrated with other systems within your state agency or within other databases within the state? That could alter or change how data is communicated in terms of the dissemination Data suppression for privacy, data suppression refers to the removal of data points to prevent the identification of individuals or groups or those with unique characteristics to reduce the risk of disclosure Suppression rules do vary by state and local area, and this makes it important to review those critical elements to make sure,

do you know what stakeholders, what audiences are accessing that data? And that could really alter what data needs to be suppressed, what’s lodging to fall on regulations related to that And also, the knowledge of your state data processes as well, do you know what processes are available in your state in terms of making your work more efficient? Who do you go to if you have questions about the publication of reports, for example, and in terms of data suppression or governance rules, and if you see something incorrect in a published report, who can you go to in order to get that corrected? Several benefits of a thoughtful process: If you think through a lot of these elements, you’ll have available and accessible data with stakeholder and leadership buy-in for them to be able to make better program decisions made from high-quality data They’ll be able to understand what’s going on with the data, understand your report, understand why it’s created that way and then be able to make better decisions out of it There’s several other benefits associated with that, of course, but definitely we wanted to highlight some of the things here And as Dan mentioned, we have several resources at IDC that are available to you should you need assistance with your public reporting of IDEA data We have checklists, for example, like the IDEA Section 618 public reporting data element checklist which will provide you with a list of elements that will help you if you’re trying to revise or rework some of your public reporting That’s something you can certainly look into We have the interactive public reporting engine which gives you some ideas in terms of how some of that 618 data can be publicly reported, and then if you want to go beyond the requirements, you also have some infographics here like the improving state reporting of local performance which is an infographic now, kind of better explains some of the information that you need to publicly report as well as the Part B Indicator Display Wizard If you’ve seen Dan or I at some conferences virtually, you may know of this wizard where you can insert a lot of your APR data to be able to show different graphs, show different visualizations of your 616 data, and if you do that, then you’ll have some options in terms of audiences, messages, things like that to be able to display that information So those are some more resources, and we wanted to go ahead and transfer to state sharing, so we wanted to first introduce Ohio, and I will pass the ball on to Kara Kara, you want to go ahead? Let’s see if I can get it to you there >> Sure Good afternoon, everyone I’m Kara Waldron, and I’m lucky enough to lead the data team within the Ohio Department of Education Office for Exceptional Children I’m here with two of my awesome team members, Matt Loesch and Ashley Rector, to show you our special education profile We’re going to us the time that we have today to highlight the data visualization features of our district profile including some strengths and challenges, and we’ll spend the second part of our presentation exploring the new section we designed for this year’s profile to capture all the disproportionality categories and requirements from the revised regulations As you might imagine, we face no shortage of challenges in that undertaking So let start by covering, what are Ohio’s special education profiles? It may be most helpful for this group to think of the special education profile as an SPP/APR that the SEA develops for every LEA based on the data reported by the LEA Only this group could follow all those acronyms in that sentence without a second thought, right? So we develop an individualized data profile for every LEA each year, and that covers over 1,000 local education agencies, including traditional districts, charter schools and state schools Though the profile is designed to display the LEA data for each of the special education indicators, it is equal parts data tool and monitoring tool The profiles show not only longitudinal data for every indicator, but they also are designed to assist LEAs in their continuous improvement planning and their monitoring activities They are our vehicles for informing LEAs of any required actions for that school year tied to the compliance indicators, the survey indicators and the result indicators We use the profiles to kick off our monitoring activities for each school year, and within the profile, indicators with required actions contain details for each step of the indicator review process,

including student-level data for the compliance indicators that have required actions So at this point, I’ll turn it over to Matt from our team to show you how the public can access our special education profiles and to show you some key features from an actual profile >> All right. Thank you, Kara Good afternoon, everybody So Ohio’s special education profiles are posted on the department website, so I’m going to share my screen and show that to you now All right So on this profile page, we have an overview of what the profiles are and the IDEA indicators that are included We also have information on how districts and the public can access the profiles, and also, we have a district-level indicator performance spreadsheet that contains mass profile, public data from all districts, although we don’t have time to look at it today So if you go to the top and click here to access the profile for any district, you can then type in the name or IRN of any district in the search box to get to their public profile summary So on the profile landing page, we include a “What’s different this year” box to highlight any additions or changes to the profile such as new business rules for calculation or new sections of analysis such as disproportionality There is information about the data year used, how to read the charts and the data that is included in the profile, and at the bottom, we have graphs comparing the district to the state in terms of the percent of students with disabilities versus students without disabilities or disability distributions So if we click to view the profile, we can see that the indicators are grouped by essential questions about kindergarten readiness, achievement for students with disabilities, post-school outcomes and IDEA timeline compliance, so just to show a few indicators as examples, let’s look at indicator 5A, least restrictive environment under the second essential question So we can see the target, the district performance data and if they met or did not meet immediately without any further click, but if we click more information, we can see a description and data notes about the indicator as well as trend data over 5 years We show the trend data on the left at the graph and in a table including yearly targets on the right So one more example we’ll look at today is indicator 11, timely initial evaluations under the fourth essential question Again, we can see the target, the district performance, if they met, and, additionally, this icon by the not met shows that the district has required actions for this indicator We click on more information again that shows all the same information as before, including the indicator description and data notes and the trend data in both a graph and a table, and one thing to note, if this was the district version of the profile instead of the public version, there would also be an action statement for the district detailing the district calculation, the indicator 11 review process and a step-by-step walk-through of the district’s required actions with links to all the required documents and details on submission protocols and due dates So I’ll end our overview by summarizing some of the strengths and challenges

that Ohio found related to data visualization So we consider it a key strength we’re able to make the profiles available to the public with the small cell sizes masked and monitoring features removed The profiles display the data visually in graphs as well as numerically in data tables, so we provide different ways to look at that data, so with up to 5 years of data displayed for each indicator, you can identify trends over time including areas of progress and also indicators that might not be headed in the right direction Finally, the profile is designed to make the quantity of data less overwhelming by grouping the indicators based on those essential questions that we saw that tell us what’s happening for kids, so we also nest the information so that it starts out in that summary form showing which targets were met and missed based on those green and red colors You can then click on more information under each indicator to see the graph, data tables and further details, so to look at the challenges next, even with the grouping and the nesting features that we showed, the sheer quantity of the indicators and the data can still make visualization overwhelming Another thing is that we don’t post the public versions of the profiles until later in the school year or even the summer, so the data do run behind For indicators that lag such as graduation and dropout, the data are even more outdated Though the profiles lend themselves into a deep-dive into an individual LEA, they don’t lend themselves to comparisons across LEAs, so that’s why we also post that spreadsheet containing indicator data for all LEAs on the profile page that I mentioned earlier Finally, incorporating the revised disproportionality regulations into the profiles has been our biggest challenge yet because of the complexity of that data I’ll turn it over now to Ashley to tell you about that >> Thank you, Matt. I will try to share my screen here Excellent. So as Matt shared, the special education profiles have traditionally been organized around those four key questions We call them essential questions This year, we’ve added a fifth to measure significant disproportionality, and this is available on the profile directly below those original four questions, so you can see here in that kind of teal bar, the essential question five asks, “Are children receiving equitable services and support?” Then directly under the essential question, you can see three gray boxes, and these boxes expand, as signified by this plus sign here So these boxes describe what significant disproportionality is and why we’re measuring it, what categories are measured in this section of the profile and finally how disproportionality is calculated into these sections that follow So as you saw, you can minimize those with what is now a negative sign in the left-hand side of this box, and then we get to the data For this sample district, we can see the three primary categories: identification, placement and discipline on the left, the target right in the center and the district’s results in comparison to the target on the right, so a high risk-ratio threshold is 3.5 for all 14 categories of analysis, so less than or equal to 3.5 is our target, and then the results column identifies where the district’s risk ratio falls relative to that target, and the status bar on the right indicates met, not met or NR as appropriate So this district is within targets for all three categories, so they see a green status bar labeled met with a good star icon For districts who exceeded the ratio threshold for 3 years and do not meet reasonable progress, a red bar labeled “not met” would display here, and there would be a small exclamation point icon identifying that the district has required actions with this indicator And the districts who do not have enough students enrolled to calculate a risk ratio, a blue bar will display in place of the green one here, and it gets labeled as NR for Not Reported, and there would be no more additional data available for this category So this layout, including the status bars and icons, aligns with the rest of the profile that Matt displayed that districts are used to seeing, so all this section and the data included here are completely visible the districts, and most of the districts, their presentation is similar to what they’re used to seeing in prior years So I know that you’re thinking, “This is way too simple and clean You cannot possibly capture all the intricacies

of significant disproportionality.” And this is actually one of our very favorite parts of this new section, and Matt touched on this a little bit It’s one of our strengths was the nesting quality of the data presentation, so this one is built to take in very small portions at one time, so those who have limited time can visit this page and say, “Oh, great We’ve read each category, and we don’t have any quarter actions,” and those with more time or just more data, as much as all of us do, they can have a choice of drilling deeper, so while this district shows met in all three primary categories, opening each category and looking at the details will help them understand if they’re at risk for significant disproportionality in key areas and then show them the trend of data over 3 years So we’ll start with the first primary category: identification for special education Before we look at the details, we can see that the district’s results field shows greater than 3.50, and this means that they defeated that threshold in one or more identification categories and that they received the status of met, so to find out why, we can click more information tab, and in the gray rows, excuse me, in this table, show the specific categories of analysis with all disability coming first The first column in this table shows the racial group, and for this sample district, not all racial groups are listed here because the district does not enroll enough students in every racial group in order to calculate this ratio The other columns show the target which is less than or equal to 3.50 in Ohio, the status which would be met or not met, which ratio type of regular or alternate and the district 10 data over 3 years as well as the plus button for more information, so we’ll look further at this district data for black students identified with intellectual disabilities, so for black students, the district met the requirement However, when we look at the 3 years of data in the small bar graph, we see all 3 years are red and above that target line, and if we hover over the little bars, we can see the risk ratio for each year included in the overall calculation, and by clicking this more information button, we can see the bar graph in more detail with the years and the risk ratios labeled This dark red horizontal line here shows the target or the threshold at which identifies the significant level of disproportionality, and in this case, the district is above that 3.5 threshold for all 3 years, so all of the bars are red Based on this district data, we would expect that their status should be not met for this category, but we still see that green met bar right there So in addition to that dark red horizontal line, you’ll also see a purple line on this bar graph, and the purple line represents the reasonable progress target based on where the district’s risk ratio started in year one which, in this case, is 16, 17 As we hover over that purple line, we can see the expectation for improvement that would allow a district to meet reasonable progress In Ohio, reasonable progress is met if this ratio lowers by at least 0.25 for 2 consecutive years There’s more information about reasonable progress in the data notes which are listed in the bullet points to the right of the graph along with the distinction of how this category is calculated, and if we scroll down below this description and data notes, we get to the data table, and this table is organized in three columns of data for each of the 3 most recent years Each row of the table details a separate calculation for each year, and if we scroll down just a bit more, we see another data table showing the calculation for reasonable progress, so the goal here is to reduce the risk for black students to be identified with an intellectual disability by at 2.5 each year This district reduced their risk by more than that each year which is identified in the green cells of the table That’s why even though we see red showing that they are above the thresholds for all 3 years, the district met the target demonstrating additional progress And because this is the actual profile and not the public-facing profile, we’ll see the action statement for this category that the district would actually see We see that this district has no required actions because they made reasonable progress, so this district would continue their efforts to reduce the risk ratio in this category, but they’re not required to submit any documentation to the department, and it’s important to note here that if the district did not meet reasonable progress, this section would contain each of their required action steps, including redirecting 15 percent of their federal special education funds to address significant disproportionality So we’ll close this category and look at one other example briefly So we’ll scroll down a bit and look at what students can be identified with autism You see that we’re looking at here the alternate risk ratio,

previous category with the regular ratio So we’ll look at more information here, and you can see that it’s laid out exactly the same way with the graph on the left and data going through descriptors on the right, and if we scroll down, you see a slightly different data table here, and this is one of our biggest challenges in working with our developer We wanted to be transparent with districts and supply the step-by-step calculation so that the districts can replicate the calculations on their own So we opted to go for a very similar table to the regular risk ratio which is here on the left, and in the right side of this table continues the alternative risk ratio You’ll see that route X in this table identifies the reason why the alternative risk ratio was required Then I’ll close these categories back up and get back to our simple screen, and building this with our developer was definitely a challenge As you know, disproportionality is very complex, but we didn’t start from scratch Everything that Matt showed we had already had built, so this section was modeled after that, so districts were used to seeing this layout Again, the nested quality of the data we really liked It was a huge strength of ours The new dispro section was also designed and refined with a lot of people’s feedback We were very fortunate to have a lot of input throughout the process Some of the strengths in addition to that alternative risk ratio calculations, especially for those districts to alternate between the alternate risk ratio one year and regular the next were kind of the delay in our release of the special education profiles this year, so as Ohio fully embraced the new disproportionality requirements because we wanted to make sure that we were collecting the data in a digestible way, the ordered school building closure in Ohio came just before we were able to release it, so while our regional staff and a lot of others have provided feedback here, we haven’t had any districts’ feedback, and this is also not yet publicly available because we were not able to release it to districts, so it’s going to be some time to practice to understand all this content that’s presented in this section, but we’re excited to get this out to the districts to make it more usable for planning their continued [INAUDIBLE] approach So if you have any questions, I know it kind of wrapped up really quickly here, but if you have questions, you can shoot them to Matt, Kara or myself or exceptionalchildren @education.ohio Cool. So I’ll pass it over to Michigan >> Hi. This is Nicholas Armit from the state of Michigan I’m just going to do some general introductory discussion for you before I hand over to Julie Trevino who works with the Office of Special Ed within the Michigan Department of Education and also Carl Jones, a senior analyst here from CEPI We’re not really going to be working from the PowerPoint presentation, but the PowerPoint presentation does provide to you some useful information about the challenges we faced when building the special ed reports as well as some of the lessons learned as we undertook that process Also discussed some of the successes we had and what we think are things to focus on to ensure a successful build of reporting such as this, but for this presentation, as I said, I will just give you some introductory remarks, and then I will pass over to Julie, and she will lead us through an overview of the reports we developed for the APR reporting, and then Carl Jones will review the next phase of the reporting that we are working on which is a special education dashboard It will take advantage of new technologies and update processing flows than we have had in the past So the special education report is a part of the state-managed website, MI School Data, which is MI School Data is a public-facing website that provides data collected by the state, mostly here at CEPI, which is kind of the data-collection agency for the ISPs, districts, schools, postsecondary institutions and other agencies in the state, but we also post data from departments within the state of Michigan and external agencies as well, so the data were then provided on the site and easy to navigate reports with both a dynamic user interface and accessible data tables for others to use The site spans all areas of education data, so it’s much broader than just special ed We have student demographic reports, and Julie will show you the front page of the site

when she’s on so that you’ll be able to see that We have testing data, districts and school accountability data, postsecondary data, and we’re also working in the areas now of workforce and CT data, so we do have a pre-K through workforce emphasis on the site It’s not just a K-12 So the site has a public-facing view which provides the general public, parents and other interested parties such as the legislature and the media, transparent views of the data, but there is also a secure version which Julie will point out to you quickly that is accessed by an approved username and password, and that section is mostly to allow district and school personnel access to review reports before they release to the public and also to enable them to review unsuppressed data for their location itself, so on the public side, reports that have FRPL implications, the data are suppressed at a certain level to ensure individual identity protection While the state site is housed and managed at the state of Michigan, when building the reports, we worked, and we still do, with a private development team that’s external to the state We worked with other state agencies The use interface code and the database structures are separated, so the external agency, the private developers work on the user interface, and the data loading and data [INAUDIBLE] are all housed internally at the state of Michigan, and then we also worked with other stakeholder groups in the education field Julie will touch a little further on how those relationships with these multiple groups had an impact on the type of reporting that was created and also on how being focused on the relationships and keeping clear channels of communication open were key to the success of the project, and we do talk about that in the PowerPoint pieces that we have as well if you’d like to refer to that after the presentation So the site was built around 2011, ’12, and the special ed reports, it was about 2013 when we got those up on the site, so some of the technology that we’ve been working with such as a software package called Telerik is a little outdated, so currently, we’re working on an extensive redesign of MI School Data for a more updated look and feel, and while this many not immediately impact the reporting for special ed, we will be working with Julie and her team in the near future to review the existing reports with an eye on redesigns and updates, so maybe we might be talking to or using some of Ohio’s ideas because they looked really great >> Nick, I apologize to interrupt you there, but I didn’t know if you were on a specific slide There was a couple people asking about >> Oh, okay. I’m sorry >> Yeah No, I’m just making sure I didn’t know what slide you were on, so I just wanted to make sure we’re on the right one >> Yeah, it’s fine So you see the … Okay I’m not a specific slide itself because Julie is going to lead us through, like, an online review >> Got you. Okay >> But there are As I said, there’s different slides that we introduced information through for audience and message, design principles, visualization, and I thought that the users could look at those as they wished to at their own leisure >> Got it >> And we’ll show some of these pieces on the review when she does that Sorry I didn’t go to that slide My apologies Let me go back to the So just to finish up, I said, we are going to redesign the site Hopefully this will enable us to update the look and feel of the site It does look a little bit 2010-ish, so we’re hoping to improve on that, but we’re also hoping that with the new software technologies available to us, which is mainly Microsoft Power BI, we’ll be able to make some interesting and useful updates to the visuals on the site and how we show the data, and some of that information you’ll be able to see when Carl does his presentation because the dashboard that we’re creating is actually created in Power BI, so Carl will be able to lead you to some of those ideas, so with that, I’d like to hand over to Julie for her presentation, and she will be pulling up a report directly, so let me pass the ball >> Thank you, Nick. Let me see if I can share my desktop here so you can see my screen So you should be able to see the that Nick was referring to, so I am the SPP/APR coordinator for Michigan in my job, and when I say it’s my job, it’s really a bunch of people,

but a lot of it is, is to ensure that we meet federal requirements of public reporting on the performance of each LEA on the state targets established in Michigan State Performance Plan Annual Performance Report of the SPP/APR and specifically for Part B indicators one through 14, so to give you a little bit background, Nick had talked about this We’ve been working together for several years, and so back when this initially started, we were kind of on a fast track to get this done, but everything was moving to the MI School Data portal, and so we had originally had the special education data in a separate platform all by itself, and so the thought was, we should have a one-stop shop so users can come to one location and get all kinds of data, and that’s exactly what kind of started this, and Nick mentioned, and I’m showing you that there’s a lot of information here Specifically, we’re going to walk through public reporting for this webinar, but I also wanted to quickly point out, we also have what we referred to as data portraits, and within this, you can access information for our students with an IEP population by disability, age range, educational setting and demographics, and the reports are all very similar, and so you can navigate through them kind of using that same process, so, again, that was very important that we make them so they were user-friendly, so I’ll show you how that works in the public reporting, so there are three tabs specifically There’s a summary, special education summary, indicator report summary and selected indicator reports We’ll start off with the special education summary, and so what we, again, wanted to make sure is that we had buy-in, and so we wanted to make sure that we had a lot of meetings with stakeholders at the time, and part of that was trying to incorporate some of the things that we had in our previous platform that were now being transferred here because we have ISD directors, local educational agencies that used this information, but it’s also a public site where, as Nick mentioned, parents used this, and other public users used this, so we have this summary, which talks about what data sets, what years this is, and so we’re actually about a week away from releasing our public reporting for the FFY 2018, and these dates will be updated, and then we also have these Excel spreadsheet links that you can download, and, again, that was part of the feedback we received from others because we had previously provided Excel spreadsheets, and there are people, believe it or not, who love Excel spreadsheets and can download these spreadsheets, and they’ll add their own data points and create their own data charts maybe for presenting to board members or for other stakeholders, and so that was something that we retained, and we have that information there We also have a link to where we have common questions so a Q and A for additional information because the most important part of this was to make sure that we’re providing users because you have, again, so many different users with different backgrounds that it was important to make sure that we are providing information so that they understand where the data source, again, because there is so much data here, and it can possibly come from So, for instance, a graduation, we report on a 6-year cohort, but in a different report, they could have a different graduation rate, so we wanted to make sure we really were clear on the data source and where this was coming You can also get additional information here So the important The tab that I just clicked was the indicator reports summary, and here, I have the indicators listed by indicator one If you go to any of these reports, you can have all state, just the state target, state performance I’ve actually, prior to this presentation, went in and loaded in one of our local school districts, Okemos Public Schools, and you can see the district performance compared to the state and then if they met or did not meet, and they did not meet You can see there’s an X for not met, and you can also just kind of see the comparison there You have about Oh I also wanted to, before I forget, I wanted to point out that you have these options of selecting, downloading a report, either PDF or CSV Again, the users may have different purposes for using this, so that has that ability to download those reports and also link to the report, and then again, when I was talking about making sure that we were transparent with users about where the data comes from, we have these links that they can click here about the data, how the data indicator is calculated,

and there’s also links to additional information about the indicators, some background on why it matters and so lots of resources here on reviewing those I can also go to the next tab, which is selected, indicator reports One of the things I also wanted to highlight was that we worked with CEPI and with Nick to make sure that these were ADA compliant because that was very important that they were ADA compliant and that they were user-friendly, so you can see in this, we have for end of the indicators, this one was specifically for indicator five You can view them all or just a specific indicator Quickly I’ll go through here, so here you just navigate through here You select a district You can do a search You can reset it to statewide, and I was looking at indicator five, but you can select all of the indicators, and here are the different years You could reset for a different year if you wanted to look at some historical View the results, and you will see that they then will come down and will list all of the indicators This text we finally worked out that most of the text stays primarily the same, but every year, we just update the performance and also the years, and then again, you can see that it’s a simple table and then a simple graph, ADA accessible, so that information is also listed, and then I wanted to point out here that Nick mentioned a log-in This is where users who have authorization can log in, so authorized users can log in and see unsuppressed data I also I know we’re running out of time but wanted to mention that here if you look at this additional When you edit the report, we have this function here where you can add a comparison, so sometimes districts, especially locals, want to see how they’re comparing or even a parent wants to compare from one school district to another, so that information is then You can add a comparison and then also remove a comparison, so I’m going to I know that was a quick review, but I’m going to transfer it now because I want to make sure that Carl has time to share his information for the changes that are coming >> Hello. Thank you, Julie Hopefully you all can hear me That was a wonderful overview of our MI School Data site I will be taking us through where we are going Okay, so I’ll be sharing the screen There’s two components I’ll be reviewing: the special ed dashboard that you heard Nick and Julie mention as well as this high-level piece on our architecture of where we’re going with using Power BI Microsoft Power BI, some of you may be familiar, maybe not It’s a business intelligence service by Microsoft It’s similar to Tableau and other business intelligence schools, but it’s roped together and create new composites of all the great data we collect and putting them together in a display Their wide range of audiences do help convey and make better data-driven decisions, so here specifically, this is a special education dashboard It was inspired by some of the work of one of the districts we partnered with where they themselves had created a dashboard and selected a variety of key performance indicators and metrics, and we thought that that wonderful work should be shared with the greater districts of Michigan so same data but produced across districts Here, I should say, the report that we’re looking at right now is using test data totally for the sake of looking at the functionality and explaining it today for you folks This landing page, we have what’s called student accounts home page, so we’ve got a variety of metrics in terms of those with IEPs, English learners, economically disadvantaged which is the equivalent of poverty in Michigan as well as numbers of graduates and dropouts with IEPs Some important terms we’ve heard floating through this presentation are things like suppression, ADA compliance and points like that, so in collecting this tool, it was important to have those considerations in mind that it could meet ADA compliance, including components such as payables and ADA-compliant colors, text box, et cetera, as well as the kinds of security and suppression pieces as well as having an interface that is easy to use, responsive, that sort of thing, so if we look at something like Allegan Public Schools here, you can see that in these 5-year trend charts, the most recent couple years, the data were less than 5 percent, so they were suppressed in this test data, of course, or not available, so having tools that can meet those needs are really important We have other components such as targets which are our indicators, so here it’s useful with these type of tools

because we can include items such as text, hover-overs, pop-ups, so it’s obviously why, in something, for example, like your indicator one graduation, why you’d want to be at or above rather compared to drop-out where you want to be at or below Those can be explained alongside the chart So these types of visuals and dynamic resources are what we hope to include and really bring forward into MI School Data, the Michigan resource, where these metrics are all available throughout the website, but tools such as Power BI allow us to bring it together to create these mosaic displays and really make them useful for the general public as well as anybody using special education data We have disability trends so the number of students within various disability categories, so, again, we have a really dynamic display, looking at statewide in this case We could look at something like, say, Detroit We’re all familiar with Detroit Public Schools most typically, and we have this really responsive, this ADA-compliant display, data tables available to help provide those type of resources and really help folks get to what they need while we’re just providing the resources and accessibility to the data Dashboard also has proficiency rates Again, right now, we’re facilitating the metrics that were inspired by what the districts have been using, but, of course, there’s always room to include more, more indicators, more of that type of thing Right now, we only have We just got English and math for third and eighth graders for their M-STEP scores, their M-STEP being the Michigan Student Test of Education Program, but we can certainly include more You’ll notice that 2018, ’19 is missing because that’s where PSAT will start, so you’ll see a break in the line chart, so Power BI is really offering us an opportunity to take the data sources, create these great models and help distribute these types of data We’ll go over briefly as well in general how Power BI functions for us, so in our data perspective, CEPI, Center for Education Performance Information, we’ve got all these great applications by which we collect data, also coursework our wonderful partners at Michigan Department of Ed as well as other departments and other data sources to create our longitudinal data system We’re also working toward the common education data standard that you might be familiar with and working toward Azure, other file-based sources, so we have all these great resources that we’ve pulled together and now using Power BI desktop specifically, getting more into the future for Power BI reporting services We work with the Power BI service, so the service is where you bring together all these data sources You’ve got your version control, that sort of thing where you manage your content, your security, suppression, make sure folks can see what they’re meant to see if looking at smaller accounts, that type of thing to protect the anonymity of the kids while providing the most value to the data, and then finally, Power BI embedded is really what Nick touched upon when his introduction to what the site is going through a redesign where we’ll be embedding Power BI, the application, into the site, so what we are just looking at is Power BI, this special education dashboard in the context of MI School Data, and that is here, but what that really is, is Power BI created through Power BI desktop, report created through desktop, which is then housed in the Power BI service and then embedded as content in the website >> Carl >> So that’s the life cycle of the process >> Yeah. We’ll have to I apologize. I don’t want to certainly don’t want to cut it off, but >> That’s okay >> I think we’re going to have to transition to evaluations, so thank you Thank you so much for coming and sharing or hearing the stories from Ohio and Michigan If you have any questions, please feel free to contact any of the state presenters or us at IDC through your IDC state liaison, and now I’ll go ahead and transfer it to Anne D’Agostino for the evaluation Thank you so much