I am back to continue and finish this three-part commentary about installing MicroStrategy v9.4.1 on my laptop.
Just also wanted to let you know that v9.4.1 Hotfix 2 was released on 02/12/2014 and is now available. I recommend you get your v9.4.1 GA version all up and running properly before you consider installing the Hotfix.
Reviewing Upgrade Prerequisites
Before you begin upgrading your MicroStrategy system, it is always a good practice to review the MicroStrategy Readme document so that you are aware of any changes from previous releases. You should also review the system prerequisites outlined in the Planning Your Installation chapter of the MicroStrategy Installation and Configuration Guide.
If you do not review the MicroStrategy hardware and software requirements before upgrading, you may experience problems with the upgrade.
Database and Driver Requirements
Refer to Certified and Supported Configurations in the MicroStrategy Readme for updated information about specific database and driver combinations certified by MicroStrategy.
System Sizing Guidelines
There are several factors to consider when you initially set up your MicroStrategy system. These factors include the number of users that will access the system, report complexity, and whether or not you should employ caches. You should periodically re-evaluate your system and update it based on actual system performance and use.
In particular, before updating your metadata (see the Update the Metadata section below), MicroStrategy recommends that you have an amount of free disk space equal to at least twice the on-disk size of the metadata database.
A complete discussion of system sizing guidelines is beyond the scope of this blog entry. Refer to the Planning Your Installation chapter of the MicroStrategy Installation and Configuration Guide for the latest details about sizing your system.
Due to performance improvements and enhancements, MicroStrategy version 9 may require more memory than version 8 for comparable functionality. In particular, if your MicroStrategy 8.x system is running on Windows and is approaching the 3 GB Windows memory limit, you may need to upgrade your Intelligence Server machines. For more information on MicroStrategy memory recommendations, see the system requirements in the MicroStrategy Readme and the Tuning chapter in the System Administration Guide.
Privileges and Access
Before upgrading, ensure you have the following:
- If you are installing on a Windows system, you must have a login account with administrative privileges for the domain or target machine.
- MicroStrategy Intelligence Server installation files. You can access the installation files from a disk or from a network location.
- Write permissions in the installation directory; otherwise the installation/upgrade process fails.
- If you have purchased a CPU-based MicroStrategy license and are installing on UNIX or Linux, you need root access permissions for installation.
- A license key from MicroStrategy for the version of the MicroStrategy software that you are installing.
Checking for Supported Data Types
MicroStrategy Analytics Enterprise supports a wide variety of data types for each supported warehouse database. However, some pre-Analytics Enterprise projects may contain data types not supported in MicroStrategy Analytics Enterprise. If a project containing columns with unsupported data types is upgraded, the data types for those columns are assigned as “reserved,” and proper data types are not assigned in temporary tables. This affects report execution.
Before proceeding with the upgrade, you must ensure that all data types assigned in the pre-Analytics Enterprise project are supported in MicroStrategy Analytics Enterprise.
See the MicroStrategy Project Design Guide for a listing of the supported data types for each database type and additional information about changing to supported data types.
Backing up the Metadata
Although the MicroStrategy installation process itself does not affect your project’s metadata, MicroStrategy recommends that you back up your metadata before any significant installation or upgrade. In most major MicroStrategy upgrades, a metadata update is required for all the pre-existing projects in your metadata. Once you update your metadata project, you cannot revert that metadata to a previous version. Therefore, MicroStrategy strongly recommends that you perform a full database backup of your original metadata prior to the upgrade.
MicroStrategy strongly recommends that you also tape backup, image, or ghost the production server before upgrading.
If you want to keep an old MicroStrategy Tutorial metadata repository and warehouse from a previous MicroStrategy version, rename the Microsoft Access files or move them to another location; otherwise, they are overwritten during the installation process. The Access files are installed by default in the MicroStrategy\Tutorial Reporting folder.
Updating the Project Metadata
MicroStrategy requires that you use the Configuration Wizard to update a metadata project created in a pre-Analytics Enterprise version of MicroStrategy to the latest version.
Be aware of the following:
- If you are upgrading a MicroStrategy 8.x metadata that is stored in a DB2 UDB for z/OS database, refer to MicroStrategy Tech Note TN32695.
- For assistance with updating MicroStrategy metadata projects from versions prior to MicroStrategy version 8.1.0, contact MicroStrategy Technical Support.
MicroStrategy requires that you update projects through an Intelligence Server connection (3-tier). Upgrading your project using a direct ODBC connection (2-tier) is not supported.
If you do not upgrade the metadata to the latest version, certain features will not work as expected. For example, if MicroStrategy Web Analytics Enterprise connects to a pre-Analytics Enterprise metadata through an Analytics Enterprise server, Change Journaling, Distribution Services, and some Report Services enhancements may not be available.
Downgrading Metadata Projects
Downgrading a MicroStrategy metadata or project to any previous product version is not supported. Once you update the project metadata to the latest version, you cannot downgrade to earlier product versions. Therefore, backing up the metadata is an essential step in the upgrade process as it allows you to revert to a backup version of the metadata, if necessary, to obtain pre-update versions of the projects it contains.
Configuring an Upgrade Test Environment
Your MicroStrategy environment includes multiple variables, such as security requirements, performance requirements, and VLDB settings, that are unique. MicroStrategy cannot anticipate all the ways these variables may interact with the upgrade process. Thus, MicroStrategy recommends you create a test environment and upgrade that environment first, then thoroughly test the upgraded installation. Once the tests are complete, then upgrade your production environment. This ensures that the upgrade of your production environment proceeds smoothly and any unexpected difficulties do not require additional downtime.
I will post a blog in the near future about testing your upgraded environment.
If you do not want to create a test environment, MicroStrategy recommends that you create and save an Integrity Manager integrity test baseline of your reports and documents. You can then execute an integrity test against this baseline when the upgrade is complete, to ensure that the upgrade has not altered any of your report results. For detailed information about using Integrity Manager to execute integrity tests, see the Integrity Manager chapter of the MicroStrategy System Administration Guide.
Best Practices for Configuring an Upgrade Test Environment
MicroStrategy recommends that you follow these best practices for configuring your upgrade test environment:
- Do not modify any existing configuration objects. If you need additional configuration objects for testing, you can either create additional objects, or duplicate an existing object and modify it. This applies to database instances, connections and logins, security filters, users and user groups, and security roles.
- If your production environment is clustered, then your test environment should also be clustered.
- If your test and production data warehouses have different database table prefixes, make sure you are using the correct prefixes in the test environment’s Warehouse Catalog.
- Create an integrity test comparing reports from the upgraded test environment with the same reports in the production environment, so that you can easily see where any differences are.
- If possible, plan to execute data integrity and performance load tests against the production warehouse. This ensures that the test scenarios are as representative of the production environment as possible.
- If you are creating reports and documents specifically for an upgrade integrity test, create those reports and documents before you duplicate the production metadata.
- If you are using connection mapping for users to access the data warehouse, check to be sure that all users can log in to the test data warehouse, since user passwords may differ between the test warehouse and the production warehouse.
One way to manage this is to create a new generic database login, and then use the following sample Command Manager script to change users’ connection mappings to use this new login:
ALTER CONNECTION MAP FOR USER “
username” DBINSTANCE “production_warehouse_instance” DBLOGIN “test_login” ON PROJECT “project“;
For steps to use Command Manager, see the Command Manager Help, or the Command Manager chapter of the MicroStrategy System Administration Guide.
- If you are using Narrowcast Server, make sure that the database copy of the Narrowcast repositories is not used when setting up the Narrowcast Server test environment. Instead, make a copy of the repositories with the Copy Repository utility included with Narrowcast Administrator and use this copy. This ensures that the test environment does not accidentally refer to a production server. For detailed instructions on creating a copy of the Narrowcast repositories, see the Narrowcast Server Upgrade Guide.
High-level Steps to Configure an Upgrade Test Environment
To ensure that your tests accurately reflect the upgrade experience, the upgrade test environment should reflect the production environment as closely as possible.
To Configure a Test Environment
- Set up the hardware for the environment. MicroStrategy recommends that this hardware duplicate the configuration of the production environment as closely as possible.
- Install your current version of MicroStrategy in the test environment.
- Using the Project Duplication Wizard, duplicate the production metadata into the test environment. For instructions on using the Project Duplication Wizard, see the Managing Your Projects chapter of the MicroStrategy System Administration Guide, or see the Project Duplication Wizard Help.
- Make sure that your test environment Intelligence Server is connected to your test environment metadata, and not your production metadata.
- If you do not intend to execute your tests against a production warehouse, duplicate the production warehouse, and ensure that the test environment points to the duplicate warehouse and not the production warehouse.
- Upgrade the test environment.
- Test the upgrade. Again, a future blog topic.
Upgrade Deployment Tests
Deploying the upgrade involves installing, activating, configuring, and running the upgrade processes for Intelligence Server, MicroStrategy Web Server, and MicroStrategy Mobile Server, as well as for the metadata, Narrowcast Server, and Enterprise Manager data repositories. These changes, as well as any other procedures that alter the production environment, should be tested when setting up the test environment.
Deployment tests should be performed by MicroStrategy administrators who normally have the responsibility of tuning and monitoring the MicroStrategy installation.
Some detailed information about installing and configuring MicroStrategy products is beyond the scope of this blog entry and can be found in the MicroStrategy Installation and Configuration Guide. The MicroStrategy Installation and Configuration Guide provides detailed procedures on installing and configuring your MicroStrategy system. It also includes important information about installing, deploying, and configuring MicroStrategy Universal products.
In addition, the MicroStrategy Readme contains information about the new products, new features, and bug fixes available in MicroStrategy Analytics Enterprise.
For detailed instructions for upgrading Narrowcast Server, refer to the Narrowcast Server Upgrade Guide.
Upgrade best practices
Review the following recommendations to help ensure the success and stability of your MicroStrategy system and projects when upgrading to the latest version of MicroStrategy.
- Follow the upgrade order and recommendations outlined in this section, in particular the upgrade checklist found at The Upgrade Process Checklist in the section below. In particular, always upgrade Intelligence Server prior to upgrading client applications such as MicroStrategy Web or Developer.
- Create an upgrade test environment by duplicating your production environment and production metadata. Upgrade this test environment and test it before upgrading your production environment.
- Do not downgrade MicroStrategy products or components on a machine to previous versions if you have already installed the most recent version of another MicroStrategy product on that machine.
- All MicroStrategy products on a machine must use the same version of MicroStrategy. Do not install or upgrade only some MicroStrategy 9.3.1 products on a machine containing older versions of other MicroStrategy products.
- Avoid installing MicroStrategy products using services such as Windows Terminal Services, which create a virtual session on the host machine. Always install MicroStrategy directly on the server machine’s physical interface, or by using a remote connection tool (such as Microsoft Netmeeting or Virtual Private Network) that takes full control of the server machine’s interface.
- If you are using clustered Intelligence Servers, then to retain stability in your Intelligence Server cluster while upgrading, shut down Intelligence Server on all nodes in the cluster before proceeding with the upgrade. For more information about clustering Intelligence Servers, see the Clustering chapter in the System Administration Guide.
- Every node in the MicroStrategy cluster must run the same version of MicroStrategy for the cluster to work properly.
The Upgrade Process Checklist
The upgrade process described in this section involves the following high-level steps. To help ensure a successful upgrade, follow these steps in the order they are presented in this section.
1. Prepare the MicroStrategy system and projects for upgrade
Preparing a MicroStrategy system for an upgrade involves reviewing information specific to your version upgrade, pre-upgrade information and prerequisites, checking for supported warehouse data types, and backing up the production metadata. It may also involve creating an upgrade test environment that duplicates your production environment.
2. Install and configure Intelligence Server Analytics Enterprise and Developer Analytics Enterprise on a test server
In this step, you install and configure MicroStrategy Intelligence Server Analytics Enterprise and MicroStrategy Developer Analytics Enterprise on a test server and then establish a connection to your production metadata.
3. Update the production metadata
In this step, you update the metadata version of your production projects using the test server environment.
4. Perform basic stability testing
In this step, you perform basic testing to ensure the stability and efficiency of Intelligence Server and your updated projects.
5. Install and configure Intelligence Server in the production environment
Once you are satisfied with the status of the latest version of Intelligence Server, and have updated the projects in your test environment, you install Intelligence Server in the production environment.
6. Install remaining MicroStrategy products in the production environment
With the latest version of Intelligence Server installed in your production environment, you now install and configure the remaining MicroStrategy products in your production environment.
7. Test the upgrade, and perform other post-upgrade tasks
After upgrading to the latest version of MicroStrategy, you perform several post-upgrade tasks such as testing the system, activating your installation, checking system licensing and functionality, managing user privileges, and optimizing your MicroStrategy system.
Next: Reviewing upgrade prerequisites
Last night, I installed MicroStrategy v9.4.1 on my laptop. I already have MicroStrategy v9.3.1 Hotfix 3 on it, but want to start experimenting with some of the new features.
I have a 10 seat license that any legitimate business can download and use for free. The link to apply and download the 10 seat version is here.
Before I talk about some of the interesting components of this install, I want to say that this was the easiest and smoothest install (actually, an upgrade) of any MicroStrategy product I have had. I installed the whole enchilada; Intelligence Server, Web Server, Mobile Server, etc.
I hope you find these notes helpful.
Impact of the Upgrade
My scenario involved the following configuration:
- I already had MicroStrategy v9.3.1 Hotfix 3 installed on my laptop.
- Windows 7 Professional with Service Pack 1
- Intel Core i7 CPU @ 2.20 GHz
- 64-Bit Operating System
- 6.0 GB RAM
- 600 GB Hard Disk
Here are a few name/product changes MicroStrategy Made:
- MicroStrategy v9.4.1 is referred to now as MicroStrategy Analytics Enterprise.
- MicroStrategy Desktop is renamed to MicroStrategy Developer.
- MicroStrategy Distribution Services replaces Narrowcast in v9.x.
Upgrading to MicroStrategy Analytics Enterprise can have a significant impact on your system. The sections below cover some of the specific effects of upgrading.
MicroStrategy Analytics Enterprise clients and servers are interoperable with MicroStrategy clients and servers from version 9.0.2 and later. However, full feature support may not be available when the MicroStrategy client and server are on different versions. To ensure full feature support, upgrade all clients and servers to MicroStrategy Analytics Enterprise.
MicroStrategy Analytics Enterprise is not interoperable with pre-9.0.2 releases. That is, clients (such as MicroStrategy Web or Developer) from MicroStrategy Analytics Enterprise cannot communicate with servers (such as Intelligence Server or MicroStrategy Web Server) from pre-9.0.2 releases, and clients from pre-9.0.2 releases cannot communicate with servers from MicroStrategy Analytics Enterprise.
If your system is using a version of MicroStrategy prior to 9.0.2, all clients and servers must be upgraded to MicroStrategy Analytics Enterprise at the same time.
All MicroStrategy products on an individual machine must use the same version of MicroStrategy. In my case, since my laptop is my only environment, I will be upgrading all components to v9.4.1. Do not install or upgrade only some MicroStrategy Analytics Enterprise products on a machine containing older versions of other MicroStrategy products. For example, if you upgrade your Intelligence Server to MicroStrategy Analytics Enterprise, and the Intelligence Server machine contains a copy of Developer, make sure you upgrade Developer on that machine to MicroStrategy Analytics Enterprise as well.
In addition, if a MicroStrategy Web client from a version of MicroStrategy prior to 9.3.1 connects to an Intelligence Server from version 9.3.1 or later, a previous version of the MicroStrategy Web search page is shown. To correct this, either upgrade the MicroStrategy Web client to the latest version, or, in the MicroStrategy Web user preferences, change the default start page to any different page, save the user preferences, change it back to its previous value, and save the user preferences again.
MicroStrategy Mobile Client/Server Interoperability
MicroStrategy Mobile clients from MicroStrategy Analytics Enterprise can communicate with Intelligence Server or MicroStrategy Mobile Server from pre-9.2.0 releases. However, full feature support may not be available when the MicroStrategy Mobile client and server are on different versions. To ensure full feature support, upgrade all clients and servers to MicroStrategy Analytics Enterprise.
MicroStrategy Mobile Server Analytics Enterprise and later is not interoperable with pre-9.2.0 client releases. That is, MicroStrategy Mobile clients and apps from before version 9.2.0 cannot communicate with Intelligence Server Analytics Enterprise or MicroStrategy Mobile Server Analytics Enterprise.
New Features and Workflow Changes
This section describes some of the changes in MicroStrategy Analytics Enterprise and earlier that may affect your users’ workflows.
For a complete list of new products, new features, and updates in MicroStrategy Analytics Enterprise, see the MicroStrategy Tech Note “New Features in MicroStrategy Analytics Enterprise.”
The default options for VLDB settings may change between releases. You can determine what VLDB default settings have changed by creating a VLDB settings report for your database type before the upgrade, and comparing it to a VLDB settings report created after the upgrade. For instructions on how to create a default VLDB settings report, see the section on Default VLDB settings for specific data sources in the Supplemental Reference for System Administration.
New features and workflow changes in MicroStrategy Analytics Enterprise
Some of the new features of MicroStrategy Analytics Enterprise that may affect your users’ workflows include:
- The name of MicroStrategy Desktop has been changed to MicroStrategy Developer.
- The following predefined security roles have been renamed:
- The Desktop Analyst security role has been changed to Analyst.
- The Desktop Designer security role has been changed to Developer.
- The following privilege groups have been renamed:
- The Desktop Analyst privilege group has been changed to Analyst.
- The Desktop Designer privilege group has been changed to Developer.
New features and workflow changes in MicroStrategy 9.4
Some of the new features of MicroStrategy 9.4 that may affect your users’ workflows include:
- In a document, if you no longer display an attribute that is used to sort data, the data is no longer sorted by that attribute. You can still select that attribute to sort data.
- When importing data from a file, the Select Linking Object panel is now the Select Attribute Form dialog box.
- When creating a new Visual Insight dashboard, the Dataset Objects panel is now opened by default.
- When creating a new Visual Insight dashboard, you are no longer immediately prompted to select a visualization type to add to the dashboard. Instead, a blank visualization is added to the dashboard and displayed.
- The menu options for adding a new metric to a Visual Insight dashboard have been reorganized. For detailed steps to add new metrics to a Visual Insight dashboard, see the MicroStrategy Web Help.
- When defining a threshold condition to display data in a visualization, to create a new metric value band, click the horizontal slider bar in the Thresholds Editor.
- When enabling a visualization to update the data displayed in another visualization, the Enable Filtering on Selection option is selected by default.
- The options to export a Visual Insight dashboard are now available in the Tools menu, under Export.
- The Graph Matrix visualization has been combined with the Graph visualization.
- The graph styles previously available for the Graph Matrix visualization (Bar, Area, Line, Scatter, Bubble, and Grid) are now available as styles for the Graph visualization.
- In the Graph panel, the Rows and Y-axis areas have been combined into the Vertical Axis area.
- In the Graph panel, the Columns and X-axis areas have been combined into the Horizontal Axis area.
- In a Grid visualization, in the Properties panel, the Fit To option is now the Width option.
- Integrity Manager now retrieves all rows of a report or document at once.
- In MicroStrategy Office, many locales no longer require the Microsoft Office Multi-Lingual User Interface (MUI) to correctly display prompt values during internationalization.
- In MicroStrategy Office, if you add multiple outline reports with dynamic grouping to a single Excel worksheet, all those outline reports retain their dynamic grouping. Previously, only the first outline report retained its dynamic grouping.
New features and workflow changes in MicroStrategy 9.3.1
Some of the new features of MicroStrategy 9.3.1 that may affect your users’ workflows include:
- In Windows, the MicroStrategy folder in the Start menu has been replaced by two other folders. The MicroStrategy Products folder contains the following items:All other MicroStrategy applications can be found in the MicroStrategy Tools folder in the Start menu.
- Command Manager
- Integrity Manager
- Object Manager
- System Manager
- MicroStrategy Web
- In MicroStrategy Web, the look and feel of the interface has been updated. A new navigational icon bar has been added to the Web interface, with options to create quick dashboards, reports, documents, and more. For detailed instructions on using this new interface, see the MicroStrategy Web Help.
- In MicroStrategy Web, on the toolbar, the floppy disk is now Save for reports and documents. Previously the floppy disk was Save As.
- In MicroStrategy Web, to share a link to an object, in your personal folder, right-click the object and select Share, then click Email Link.
- Visual Insight analyses are now referred to as quick dashboards.
- In Distribution Services, the Use Send Now privilege is no longer required to send a preview of a subscription. The new privilege Use Send A Preview Now is now required to send a preview of a subscription.
- The MicroStrategy SDK is no longer available to be installed with the MicroStrategy release. The most recent version of the MicroStrategy SDK is available as a free download from the MicroStrategy support site https://resource.microstrategy.com/msdz/default.asp.
- Update packages can now be hosted on remote servers. Prior to MicroStrategy 9.3.1, update packages were required to be hosted on the Intelligence Server machine. For information about importing update packages, see the Managing Your Projects chapter of the MicroStrategy System Administration Guide.
- If you are upgrading from MicroStrategy 9.2.1m or earlier, some widgets that were previously created to display on Android tablets may display as grids or graphs on the mobile device. To display these widgets correctly on Android tablets, see Updating Android widgets from MicroStrategy 9.2.1m.
Next: Upgrade Best Practices
This is an important read for anyone who works in the data visualization profession. I ask you to be reflective as you read this. I use myself as an example of what not to do.
For the past two years (2013 and 2014), I have submitted an entry into the MicroStrategy World Dashboard Contest. In both years, I was named one of the winners of this competition. I have written about my work and about the competition on this blog. But I did not tell the whole story. I did not mention that my entries were reproductions of the original ideas and designs of other people. I took liberties that, at the time, seemed innocent. As an academic pursuit, I attempted to recreate these originals using a different tool set (in these two cases, MicroStrategy Report Services and the Visualization SDK) to see if it could be done. I spent time trying to develop methods to allow me to recreate the original visualizations almost exactly as their authors had idealize and developed them. I meant no harm, but what I did was wrong.
There are lines and sometimes we cross them. There are lines and sometimes we don’t see them. There are lines that are bold and there are lines that are blurry. The line that I crossed appears bold in retrospect but was blurred at the time. I had spent a considerable amount of time developing these visualizations. It is quite possible that I spent more time trying to recreate the original than the author spent developing the original.
I know now that this does not matter.
I took the ideas and content and submitted it as my own. I am sorry for this and I have learned a great deal as a result. I now want to use this discussion as an example for others.
2013 MicroStrategy Dashboard Contest
My entry in the 2013 Dashboard Contest was a Student Performance Dashboard, which was based on portions of the top three entries in Stephen Few’s Dashboard Design Contest that was held in late 2012. The majority of my dashboard was based on the original design of the first place winner, Jason Lockwood, who had developed his dashboard in Photoshop.
At work, several of us were talking about Jason’s winning entry and how you could probably develop it fairly easily in Tableau, but probably not so easily in MicroStrategy. Being a strong proponent of MicroStrategy, I argued that I could develop that exact dashboard using MicroStrategy’s Report Services and their Visualization SDK. My co-workers challenged me to try it and I began my mission. Unexpectedly, MicroStrategy soon announced their 2013 Dashboard Contest and I thought this would provide me additional motivation by developing the dashboard for their contest.
Back to Stephen’s contest. Back in August of 2012, Stephen Few, data visualization evangelist and author of the seminal book, Information Dashboard Design, announced a contest to design a dashboard following best practices and principles. The contest required participants to design the dashboard using student performance and assessment data that Stephen provided. Any graphic design tool (e.g., Photoshop, InDesign and Excel) or BI tool could be used to create the dashboard.
The winners were announced in October of that year. There were 91 entries. The contest focused more on innovative dashboard design principles rather than the use of BI tools. The winners and the tool they used are:
1st Place: Jason Lockwood Photoshop 2nd Place: Shamik Sharma Excel 2010 3rd Place: Joey Cherdarchuk Excel 2010
To the best of my knowledge (and Stephen’s), none of the 91 participants in the contest used MicroStrategy to create their dashboard. A few of the participants did use Tableau and SAS. This fact alone made me want to create an innovative dashboard to demonstrate the capabilities of MicroStrategy.
Below are examples of the first, second and third place winners entries.
Jason’s entry (first place)
Shamik’s entry (second place)
Joey’s entry (third place)
Below is a screenshot of my entry developed using MicroStrategy and Stephen’s sample data.
As you can see by comparing my dashboard to Jason’s. I tried to follow Jason’s entry very closely since my goal was to reproduce his entry as close as possible using MicroStrategy.
I have emphasized the word “reproduce” because in my goal to prove the capabilities and functionalities of MicroStrategy, I now realize, in retrospect, that I crossed a line in using Jason’s original idea, design and work to create my dashboard. Now, if I was doing this in my basement for my own edification and learning, that probably would have been o.k. since it was not being viewed by a public audience. However, when I entered the dashboard in MicroStrategy’s contest, albeit developed using my own skills in MicroStrategy, I was presenting someone else’s original ideas and design work without their permission. This, I now understand, was wrong.
I have had several e-mail conversations with Professor Alberto Cairo about this. Alberto is considered by many (including me) to be one of the industry’s leading experts on infographics and a person I respect and view as a mentor. I was seeing grey areas in what I had done where Alberto was correctly seeing things more in black and white.
Below are some of Alberto’s thoughts on what I did and some analogies he made. I have included his comments completely in quotes to indicate these are his thoughts and have not been modified by me at all.
“There are not really clear-cut rules about plagiarism in visualization in infographics, which is a shame. It’s an area in which a lot of thinking and writing needs to be done.
But when doing ethical reasoning you can always use analogies. When in doubt, imagine that your graphic is a news article or a research paper. Would it be appropriate if anyone took what you wrote and then just make it interactive without getting permission from the author (you) first? Would it be enough to mention you in a description of what was done? It wouldn’t. Quoting a few lines from someone (in between quotation marks) is fine. Copying and pasting paragraph after paragraph is not, if it’s not without proper permission.
In visualization, things get really tricky sometimes. For instance, if someone creates a simple bar graph based on ten data points, do I need to get permission to create a similar graph? Probably not if a) the graphic form is so common, b) I can have access to the underlying data. But when you copy an entire layout, or an unusual graphic form, then things become problematic. Again, going back to my analogy before, it’d be equal to copying an article, a newspaper story, or a blog post. Even if you mention the source, it’s not something you can do without asking for permission. It would be a clear case of plagiarism, and it could even get you into legal trouble.”
Now, I take full responsibility for what I did and apologize to Jason, Shamik, and Joey. I do need to say, my primary purpose was to create recreate cool dashboards or infographics I had seen, in my tool of choice which is MicroStrategy. The key thing I was trying to do was show clients and business partners that I could create the same thing they see in Tableau and Qlikview using the MicroStrategy platform.
To continue with Alberto’s thoughts on this, I again include an exact quote of what he said.
“I understand it, but copying the layout, the structure, the content, and even the headline and intro copy (on top of everything else) is not the only issue, but also submitting the results to contests with no permission from the original authors, and without mentioning them.
Again, analogy: Imagine that I take one of the wonderful posts you have written about historical visualizations –some of them are indeed great,– and I reproduce it with no permission from you, but I casually attribute it to you once: “Hey, I’ve just found this great post in Michael’s website; I’m building on top of it, adding some pictures, and making it interactive.” You’d certainly feel uncomfortable if I didn’t contact you first. And you’ll probably get really upset if, besides that, I get a writing award thanks to that post (without mentioning you,) to which I just added a few visual elements, and interaction.”
Alberto is correct. I would be upset too.
2014 MicroStrategy Dashboard Contest
My entry in the 2014 Dashboard Contest was An Exploration of Tax Data. It was based on an original idea, text and design by Jim Uden, one of my classmates in Professor Cairo’s MOOC course on Data Visualization and Infographics.
I really liked the An Exploration of Tax Data visualization created by Jim. I liked it so much in fact, that I wanted to make a working example for our development team at work using MicroStrategy. I create a lot of dashboard “templates” for our development team in MicroStrategy, which is our enterprise standard BI tool.
So, using Jim’s data, text and format exactly, I created a dashboard in MicroStrategy with some tweaks to it.
Below is a screenshot of Jim’s original work.
Below is my version created using MicroStrategy Report Services and their Visualization SDK.
I used horizontal stacked bar charts instead so that the viewer can visually see how social security and income tax rate add up to the total and explains visually why the countries are ordered the way they are on the dashboard. I also separated out $100K and $300K percentages into separate visuals.
In addition, I added the flags of the countries.
Now, you don’t see any numbers on the data points in this dashboard. The reason you don’t see them is because they appear when you mouse over a bar where you then see the country, category and the percent value as a tooltip.
However, by using Jim’s data, text and design exactly from his original, and without getting his permission first, I again crossed the line. I have emphasized the word “exactly” because in my goal to prove the capabilities and functionalities of MicroStrategy, I now realize again in retrospect, that I crossed a line in using Jim’s original idea, data, text, design and work to create my dashboard.
I also discussed this with Alberto and his comments were,
“If this were just a class project for the MOOC, you should have asked for permission from Jim Uden, but I don’t consider it a huge ethical problem. After all, when you submitted it to the forums, you mentioned that it was an interactive version of Jim’s project, and you thanked him publicly in your message. You didn’t let him know about this directly, by contacting him (which is, again, the appropriate thing to do,) but you were transparent when you credited “Jim Uden” for the original idea. The true ethical problem arises when you didn’t do the same in your post about the exercise in your blog, and when you submitted it to a contest.”
I again take full responsibility for what I did and apologize to Jim.
I approached Professor Cairo again with another question: What are the ground rules for the use of another person’s materials. For instance, a lot of blogs (including mine) will post an infographic they have seen in a magazine or on a site like visual.ly and discuss it. Are we plagiarizing if we cite the author, magazine, etc.?
Professor Cairo responded,
“As for (this) point, there’s something called “fair use” in US copyright legislation. It’s quite fuzzy and controversial, but it basically says that if you reproduce a piece of art just to comment on it or to review it (not to build on it or to change it, not to get profit from it, etc.), you are fine. Academics and bloggers do this all the time. However, some media organizations are known for having asked bloggers to withdraw images of graphics in the past. They have the right to do so, although I think that it’s a bit silly.”
I feel this was an important topic for me to discuss and clear my conscience. I would not be honest if I did not say this was very difficult and embarrassing to write. Professor Cairo reminded me it takes courage to do this. Maybe so, but I don’t feel very courageous at the moment.
Next year, if MicroStrategy has another Dashboard contest, I plan to create the entire thing from scratch. Data, text, design, colors, fonts, etc. It will be from my vision only. However, over the next year, I think this is an important topic to discuss in our data visualization community with social media like Facebook, Twitter, Reddit, Instagram, Tumblr, etc. growing in use every day. At what point have we crossed the line? Or are they blurred lines?
I would love to hear your thoughts on this and opinions. I may not like what I will hear, but I will hear and reflect on what you have to say.
Thank you for reading this very long post. I hope you see the value in it as much as I do.