>I find this conversation interesting and wonder if you could discuss the “why” of the standards?

This is an interesting question which I am daunted at the prospect of answering in a real way as opposed to giving a glib answer. Philip Howard wrote a post recently that speaks to some of the data reasons for a common data standard or data model.

I think there are some important considerations that make standards important. The following five points were identified by the State Chief School Officers as critical to the nation’s data needs (thanks to Chris Lohse, Director of Data and Research for his articulation of these). These are in my opinion the reasons why we need data standards.

1. Liquidity – Data liquidity is by far the most discussed challenge around education data. Today, the term has gained much more traction in the context of healthcare and patient records, but the fundamental concept as applied to education is that a student’s data should move with him or her and should not be bound to a particular school information system or district data system. As one healthcare commentator noted, it is after all, the individual’s data.

2. Comparability – We struggle greatly in education with developing common definitions for even the most straightforward of education terms. During a workshop in Toledo, OH, participants at six different working tables were asked to write their definition of ‘high school graduation rate.’ Participants were then asked to pass their definition to the next working table and have the participants at that table interpret the new definition just handed to them. The results demonstrated 1) nobody had the same definition of ‘high school graduate rate’ and 2) nobody agreed with the definition that was shared with them. Until common definitions for even our most basic educational outcomes are defined, we will be greatly limited in our ability to actually understand what is happening among all students across the country – and even across a state or school district.

3. Relevance – Schools and districts collect swaths of education data but much of what is collected has limited value to teachers, parents, students, and school leaders. More often, the data are merely tracking outputs, rather than an actual change in status or condition. For example, under No Child Left Behind, many saw limited value in knowing whether teachers were highly qualified according to certifications, tenure, and degree attainment. Rather, a more relevant outcome and metric would be a gauge of whether teachers were effective in terms of student growth.

4. Timeliness – Discovering that a particular student is behind their growth and learning trajectory at the end of a school year, with little to no time to change course, is of limited value. Instead collection, analysis, and reporting information about students, teachers, schools, school leaders, and districts should take place on a time schedule that allows for change of course and performance improvement. Data and analysis provided beyond the point when corrective action can be taken is, at best, useless. At its worst, it is irresponsible.

5. Accuracy – The strategy and analysis emerging from good information is only as good as the underlying data. Increasingly, self-validating data collection tools are improving data accuracy. Put simply, when relevant data are presented to teachers and other education stakeholders in a timely, coherent way, those stakeholders begin to ensure the quality of the data inputs.

>Are they to define or align expectations in US education?

This is a key thing for us to remember- there are different kinds of standards. There is work ongoing with 48 states to create a comparable, standardized set of student expectations. These are academic content standards. I am primarily talking about data standards.

These are more about defining data elements, their data type, their length, their description, their relationships with other data elements. For example – Student First Name, Course ID, Enrollment Status, etc. I would call these “data standards”.

There is a new effort coming under way which I will be involved with and documenting closely to set data standards in the country. This is being done in partnership between USED, CCSSO, SHEEO

>If so, in what realm – student expectations, teacher expectations, system expectations?

Yes – the direction is to both support next generation learners and teacher development. System expectations are directly correlated to that. I don’t have space right now to discuss this but it is happening.

Another question that came to me from a discussion with Jean around some pilot work in healthcare – these pilots have created a hospital in which doctors aren’t compensated in a fee for service format, but rather by salary with bonus incentives around patient outcomes.

I began to wonder if there is work going on to define standard desired outcomes for school systems and if so, if there are any areas where teams of teachers and administrators are given an incentive bonus for particular student outcomes?

This is a huge issue. I won’t attack it in this post and much has been written about it, but it is clear that teacher effectiveness and accountability is going to be massively focused on and it is- I believe- a general consensus that some kind of growth model is necessary to do any kind of reasonable analysis of effectiveness of students, educators or systems in education.

Enough for today- more later. Thanks for the questions!

“You keep using that word. I do not think it means what you think it means!”

There are many kinds of standards. To begin a conversation for, ahem, standardizing on some we need to start by determining what kinds of standards we are talking about. In this post I will start listing some standards. I started thinking about this topic newly when my friend BethAnn from Virginia’s Department of Education disturbed our post-conference-session drinks with the question of “What do we mean by standards?”.

There are many ways to think about that. One way is to think about “de facto” standards versus “de jure” standards vs specifications. I think it is important to recognize which ones we are talking about.

“De jure” standards are standards set by law, by some authoritative body that has a mandate and the power to say how it goes. In the National Data Standards movement we need to be cautious that we are clear that we are not creating a mandate but an agreement by consensus. If people perceive that the United States Department of Education (USED) is telling them how they need to do their data systems, many will dig their heels in and it differently only because they can. On the other hand, if USED does not take a stand and display leadership then many will not take it seriously. It is a difficult balance we will need to strike. The “http” and IEEE standards are good examples of technical “De Jure” standards as are the various state’s academic content standards.

In fact “De facto” standards are what we need- standards that arise out of use and agreement as THE way to do something not out of an authority but rather out of consensus. That, unfortunately, will require a great deal of communication and more bandwidth between users, state education agencies (SEAs), and local education agencies (LEAs), and vendors than exists right now. The Schools Interoperability Framework (SIF) specification has become in fact, in many ways, a “De Facto standards for interoperability in the k12 space, as the Postsecondary Electronics Standard Council (PESC) has become for post-secondary education. The NCES Handbooks are another example of this kind of standard.

Specifications are “a” standard way to do something rather than “the” way. SIF, PESC, and Dublin Core are all examples of specifications. These are easier to pull together but less likely to get adoption because of the lack of authority. This gets blurry when some states start to mandate SIF for example and thus essentially turn the specification into a “De Jure” standard.

So that is one lens to view standards through. I wonder what the dictionary has to say… according to Merriam-Webster Online 2009:

Etymology: Middle English, from Anglo-French estandard banner, standard, of Germanic origin; akin to Old English standan to stand and probably to Old High German hart hard
Date: 12th century

Interesting – so we need to stand hard.

1 : a conspicuous object (as a banner) formerly carried at the top of a pole and used to mark a rallying point especially in battle or to serve as an emblem

It could be argued that this conversation is a rallying point in the battle for education reform!

2 a : a long narrow tapering flag that is personal to an individual or corporation and bears heraldic devices b : the personal flag of the head of a state or of a member of a royal family c : an organization flag carried by a mounted or motorized military unit d : banner 1

Unfortunately some of the players jockeying to control the National Standards Movement have this in mind rather than the good of education in this country! No, dear reader, of course I am not talking about you!

3 : something established by authority, custom, or general consent as a model or example : criterion
4 : something set up and established by authority as a rule for the measure of quantity, weight, extent, value, or quality

Pretty clear.

5 a : the fineness and legally fixed weight of the metal used in coins b : the basis of value in a monetary system

Nope. Not this one.

6 : a structure built for or serving as a base or support

This is my favorite- we are building the national data standards as a structure built for or serving as a base or support.

7 a : a shrub or herb grown with an erect main stem so that it forms or resembles a tree b : a fruit tree grafted on a stock that does not induce dwarfing.
8 a : the large odd upper petal of a papilionaceous flower (as of the pea) b : one of the three inner usually erect and incurved petals of an iris

Ahem. I really didn’t know what to write about this one! We certainly don’t want to induce dwarfing in our education system.

9 : a musical composition (as a song) that has become a part of the standard repertoire

We can all sing along!!!

Okay. That was fun. But that isn’t even the end of it. There are also different domains of standards.
SIF is an interoperability standard – it creates a blueprint describing how educational data can be moved from one system to another. PESC is the same except its scope is postsecondary, and SIF’s scope is K12.

The NCES Handbooks and the National Education Data Model (NEDM) are content standards. Although NEDM is more of an aggregation of all the standards and data sets out there than a standard itself.

The standards about what a student must know, or the assessment benchmarks that a student must hit are curriculum or academic content standards.

There is a lot of confusion about these different domains and when we are speaking in this national dialogue it is important that we are clear which domain, type and definition we are using.

It is my commitment that we will do this – and that what we build together as we start to build the National Data Standards is clear and that our approach is, well, standardized.

More later. Stay tuned.. same standard time, same standard channel.

The answer is simple and complex:  time and space.

First, space: “vertical reporting” refers to the movement of data between organizations at different levels of hierarchy in their area.  For instance, many schools belong to a district, many districts belong to a regional service center, and many regional service centers belong to a state (some states skip the regionals).  Vertical reporting is the movement of data up that chain of organizations:  school data go UP to district which transmits them UP to the state.  Sometimes people will talk about “vertical interoperability” which means data can go both ways.

Second, time: “Longitudinal reporting” refers to the reporting, acquisition, and dispaly of data that have a time element to them.  It is often represented by graphs.  For instsance the change in enrollment over the last ten years would be an example of longitudinal reporting. Longitudinal reporting is the reporting of data with a time element attached and used as a dimension.

When people refer to longitudinal data systems (as in LDS Grants) they are referring to systems that almost always have a vertical and a longitudinal component to them.

At one level that is a no brainer.  Of course that would be true.  But we don’t act that way.  We act as if the way it occurs for us is the truth.  An example of this came up at the National Education Data Conference in Seattle last week.    Someone mentioned to me that a panel of people had spoken to all the states who had recieved longitudinal data grants from the government and had shared how SIF didn’t work.  They spoke about it like an “of course”.  Other states though were walking around talking about how SIF was revolutionizing their data quality and data collections.  How could they both be true.   I have heard these extreme positions multiple times.

What became clear to me is that people were speaking, and complaining, based on the way the world occurred to them.  It isn’t that they were “wrong”, though I was apt to leap to that conclusion,  they were just speaking their “TRUTH”.  What is so is that SIF is working in hundreds and hundreds of districts in a horizontal (which is to say local to a school or district) or “classic” deployment, and is working statewide in a few states in a vertical manner, and half a dozen other states are working on it.  So why the bad press?

I say it is because the problem that SIF is brought in to solve is complicated.    Building enterprise architectures out into deployments that scale multiple levels of organizations (school, district, regional, state),  broad breadth (all the districts, all the schools) , and heterogenous applications (Student Information Systems of multiple types, Transcript brokers, state data warehouses, data collection tools, etc.) is complicated stuff.  The data flow inside any one of these components is complicated enough.  Managing the data flow through and around them all is hideously complicated.  Thus the issue.

Most states are not approaching the problem with an appropriate respect for that complexity.  They buy a product (like an off-the-shelf “data warehouse” or similar product)  and expect that to solve the problem without really mapping out their issues, use cases, data architecture and process flows and then putting together a system that works.    Then when the system falls inward on itself or doesn’t hit its milestones it becomes “SIF doesn’t work”.  The truth is  enterprise system design is hard, and almost impossible to succeed at in a political, consensus-driven environment.    There are so many points of failure and SIF- in automating the processes- reveals those breakdowns and issues, and, as with most messengers, often loses its head.

When you want an assessment of something make sure you are asking someone who understands those types of problems and has a track record with them.   The person who has never succeeeded may just not know what it takes, and then they will cry to the world “Don’t do X”.  If we can understand the root cause of the failure suddenly the world will appear a different way and then our actions will be correlated to that new view.

Now, telling who the real experts are and who the savvy sales people who memorize the jargon du jour… that is another problem entirely!

(1) a state department of education I am working with

(2) my own organization and our virtual team

(3) a consortial organization I belong that sets national standards (go ahead- guess! Yes- it is the SIF Association )

One of the issues everyone in  executive or management roles deals with is how to produce the results that keep the enterprise alive and keep the people that make up the enterprise interested, productive, satisfied and acting as owners of the enterprise.

I am going to proceed over the next six months to study, examine, and experiment in this area of thinking in all three of these areas of my work life.  I am not sure what it will look like or what I will come up with but I think it will be interesting and should have a positive impact on all three areas.

I decided I needed a frame for my thinking so I am going to use the context of a leadership book I have been reading an advanced copy of called _The Three Laws of Performance_ by Steve Zaffron and Dave Logan.  I will be applying the “Three Laws” and their corrollaries to each of these situations above and the implementing practices t oexecute those theories and get feedback as to the result.    I will use this blog as a forum to discuss my results.   BTW: I highly recommend reading this book.

I will tag posts with “Department of Education”, “SIF”,  and “Management” when dealing specifically with those areas.    I will tag all my posts in this strand with “The Three Laws of Performance”.

I will start my first posts next week sometime.

I am going to not address tools that would be purchased singly by a teacher or school, but, rather, enterprise solutions- with economies of scale- that would be implemented at the district or state level.

Here is my initial list.   I hope you are listening Arne Duncan!!!

1. Flexible, SIF-enabled, unit-record-level(student, staff, course/section, building, district),  data quality-enforcing data collection systems.

2.  Semantically-tagged, multi-purpose, longitudinal, flexibly periodic, data storage

3. Enterprise Directory-enabled Portal solutions that provide authorization down to the parent, student, teacher level and provide security for both access to applications and resources but also constrict viewing parameters on reports.   Centralized management and indentity services but decentralized control

4. Inside these portal solutions Communities of Practice (data providers, teachers, families, program providors),  with collaboration and file sharing capabilities as well as access to reports designed specifically for them, and workflow tailored for them.

5. Learning Standards Repository and National Data Model and NCES Handbooks all available in referencible form and allowing free access to the date through a variety of APIs.

6.  Working with SIF and PESC and others to construct Transcript standards  and templates for the whole country

7. Mature the SCED specification to include elementary school and more metadata.

8. Mature the SIF specification to provide more value and out of the box interoperability to systems all through pk12.

9.  Provide Best Practices for maturing systems that are not driven by a single solution-providor but by healthy, standards-linked functionality.

We did quite a bit of good work resolving some technical issues with the XML Schema of version 2.3 of the SIF Specification release that is now finally coming close to release.  We also dealt with some of the details of re-designing processes and growth as the organization continues to grow and add new staff (7 now) and new members.   We are close to compiling all the charters that will define the scope of the Columbus release.

It was exciting to welcome Kevin Harrison from Virginia, David Holt from Wyoming, and Matt Howard of eTech Ohio as new members to the Tech Board, and Mark Reichert from CPSI, Ann Savino from eSchoolData, and Jason Wrage (albeit  temporarily) as new members of the Board of Directors.

Looks like Andy Elmhorst of EduStructures and myself are still the co-chairs of the Technical Board.

I don’t know yet what I am going to say!