Participatory action research about Figshare user experiences at the University of Melbourne

Authorea preprint November 01, 2017 DOI: 10.22541/au.150954885.58217149

Participation & feedback are welcome! Please email me on cobi.smith@unimelb.edu.au (which is treated as private unless you explicitly consent to sharing) or tweet @cobismith (public) if you'd prefer not to comment on this working paper using Authorea's features.

Please note this is an open notebook and is intended to be part of an open science research project, which means if you choose to share information here your contributions are in the public domain. See the University of Melbourne research protocols for more information: http://www.orei.unimelb.edu.au/content/when-approval-needed

#Introduction

http://researchdata.unimelb.edu.au/figshare

http://researchdata.unimelb.edu.au/figshare/features-roadmap

http://researchdata.unimelb.edu.au/figshare/figshare_comparison_table

these linked texts were written by @GoodContentKel in collaboration with the University of Melbourne's Figshare working group.

Methods theory

Participatory action research

a spiral of steps, each of which is composed of planning, action, observation and the evaluation of the result of the action... collective reflection by participants on systematic objectifications of their efforts to change the way they work (constituted by discourse, organisation and power relations, and practice). (McTaggart 2006, p315)

Action research has an individual aspect - action researchers change themselves, and a collective aspect - action researchers work with others to achieve change and to understand what it means to change. Action research involves participants in planning action (on the basis of reflection); in implementing these plans in their own action; in observing systematically this process; and in evaluating their actions in the light of evidence as a basis for further planning and action, and so on through a self-reflective spiral. In deciding just where to begin in making improvements, a group identifies an area where members perceive as cluster of problems of mutual concern and consequence. The group decides to work together on a thematic concern but to change things they must confront the culture of the institution (or programme) and society they work in. (McTaggart 2006, p317)

Progressive inquiry

Distributed expertise is a central concept in the model. Progressive inquiry intends to engage the community in a shared process of knowledge advancement, and to convey, simultaneously, the cognitive goals for collaboration. Diversity in expertise among participants, and interaction with expert cultures promotes knowledge advancement... (Muukkonen, Lakkala & Hakkarainen 2009, p3714)

The process begins by creating the context to anchor the inquiry to central conceptual principles of the domain or complex real-world problems. The learning community is established by joint planning and setting up common goals. It is important to create a social culture that supports collaborative sharing of knowledge and ideas that are in the process of being formulated and improved. (Muukkonen, Lakkala & Hakkarainen 2009, p3714)

User experience

http://tfa.stanford.edu/download/TenUsabilityHeuristics.pdf

Currently there are three main approaches to applying and interpreting user experience in
design. These are the measuring approach, the empathic approach, and the pragmatist
approach. The role of emotional experiences is important in all three, although, as they
stem from different disciplines, they treat emotions differently. (Battarbee & Koskinen 2005, p6)

These three approaches propose divergent methodologies for studying user experience,
but imply different things. The measuring approach focuses on emotional responses, the
empathic approach on user-centred concept design, while the pragmatic approach links
action to meaning. The measuring approach is useful in development and evaluation, but
is more difficult to apply at the fuzzy front end of design (Cagan and Vogel 2002). The
pragmatist approach concentrates on the embodied nature of experience and interaction. (Battarbee & Koskinen 2005, p7)

Value can be broken down into specific attributes that contribute to a product’s usefulness, usability, and desirability, and connect a product’s features to that value. Since products enable an experience for the user, the better the experience, the greater the value... (Cagan and Vogel 2002, p62)

However, all these approaches are individualistic, thus
missing a crucially important aspect of human experience. People as individuals depend
on others for all that makes them truly human. Experiencing happens in the same social
context—therefore, it is necessary to account for this context and its effect on experience. (Battarbee & Koskinen 2005, p7)

Both social consciousness and social interaction provide opportunity for added value in a product. (Cagan and Vogel 2002, p66)

Codesign

Lesson 3: Open up the study as a community event in order to attract participants from widespread backgrounds...

Lesson 4: Allow peers to help one another to encourage participation and observe learning dynamics...

Lesson 5: Create opportunities to observe social dynamics at different levels within the community. (Ramachandran et al. 2007)

##Open notebook

Learning about a science ideally involves a process of "enculturation" into a community of its practice... effort to reason purposefully in collaboration with others, especially in writing, provides the best possible motivation and occasion for articulation. (O'Neill & Gomez 1994, p420)

##Open innovation

The patent system is built on the premise that exclusion leads to innovation. But a mounting body of evidence calls into question the assumption... (Chien 2015)

Inclusion

a barriers approach may perpetuate social exclusion. By locating the cause of exclusion with structural issues science communication institutions and practitioners can do little about, such as location or poverty, or with participants’ attitudes and behaviours, questions about whether science communication practices are themselves problematic can be deftly side-stepped... Developing inclusive science communication practices may require critically assessing current practices, perspectives and motivations in combination with a concerted call to action that places equity at the heart of science communication, rather than on the periphery. (Dawson 2014, p3)

Australian cultural context

That our science has indeed delivered benefits, not only in terms of industry innovation, but also in terms of fulfilling our own personal aspirations, there is little doubt. Whether such benefits are being delivered cost-effectively or in areas of chief concern to clients of research are more the questions currently being posed by those who are questioning science’s role in the community. (Carberry 2001)

Methods Practice

Document notes & findings from #mozsprint specifically related to the new UniMelb & Figshare collaboration:
https://etherpad.mozilla.org/unimelbfigshare

Participatory action research snowballing from a potential group of users from initial sprint. Snowball sampling led to participation of Andrew Lonsdale @lonsbio, via Cobi's participation in a Combine network meetup (http://combine.org.au/) 11 June 2015.
Add Neil's input here too from email. Add #hackyhour references & check consent for documentation from other meeting spaces.

Retrospectively understanding how Figshare @ UniMelb came into being & discussing with those stakeholders & current working group their participation in project documentation.

Next steps:

  • Ask people: are focus groups a next step, and if so should they be based on research area or user type? (in progress) (timing of user testing depends on road map: http://researchdata.unimelb.edu.au/figshare/features-roadmap
  • Ask the working group & pre-study stakeholders whether they consent to being acknowledged by name in the open notebook (in progress)
  • Explore university norms and tools for documentation of user experiences (in progress)
  • Establish links with existing data management, open science & informed consent knowledge bases across university (in progress)

If while reading these next steps something comes to mind that might be relevant, feel free to email me: cobi.smith@unimelb.edu.au, comment on this working paper, or tweet @cobismith.

Results: open science and documentation of the project

Some participants are choosing not to be documented as part of the project, or are unintentionally excluded from research due to limitations related to time, capacity, proximity, resources - other factors such as systemic biases should not be excluded (Dawson 2014). This project is a work in progress - feel free to email cobi.smith@unimelb.edu.au or comment on this article should you wish to participate!

Some who have not explicitly consented to be documented & identified at this time (9 so far, beyond mozsprint pilot) nonetheless invested time in sharing their perspectives and experiences. Articulated reasons for non-participation in documentation included:

  • uncertainty about future impacts of documentation of their views at this point in time;
  • preference to view existing documentation before deepening their level of participation;
  • uncertainty about what open science means;
  • concern about perceptions they were investing time in this at the expense of other projects;
  • greater caution than usual given emphasis on informed consent.

An anonymous contributor expressed interest (corrected from an assumption by Cobi about feeling of concern to feeling of interest) in the limitations of informed consent in open research. They consented to documentation of their interest, saying "you have my consent, but not my informed consent".

People beyond this documentation were present during in-person discussions at the Mozilla Science Lab Global Sprint and participated as listeners, sometimes interjecting with their thoughts and expressions that were undocumented.

Results: user and non-user types

Initial focus of the research has been to establish whether there are meaningful categories of use cases and user experiences. Several groups of potential and actual users, as well as non-users, were documented during the 2015 Mozilla Mozilla Science Lab Global Sprint hosted at Research Platforms within the University of Melbourne. Use cases span disciplines - later research will explore use cases from particular research areas. Firstly, user experiences of Figshare were explored.

Categories of users and non-users represented in the initial exploratory documentation during #mozsprint were:

  • researchers from UniMelb who hadn't heard of Figshare but might be interested in piloting it
  • researchers from UniMelb who have heard of Figshare but never used it but might be interested in piloting it
  • researchers from UniMelb who have heard of Figshare, tested it in the past, but not shared research objects
  • researchers from UniMelb who already have been using Figshare individually
  • non-researchers from UniMelb who hadn't heard of Figshare who might engage with content accessible through melbourne.figshare.com
  • researchers external to the university who might engage with melbourne.figshare.com
  • non-researchers external to the university who might engage with melbourne.figshare.com

as this project progresses and melbourne.figshare.com develops, people may change what category represents their user experience.

All documentation from the etherpad below needs to be better articulated via narrative:

  • researchers external to the university who might choose not to engage with content accessible through melbourne.figshare.com
    (please add your details here if this is you and you're happy to be contacted!!)

! external people who might engage with content accessible through melbourne.figshare.com

@spetznatz
"the trend is towards open science - that's because of young people primarily because they understand the potential of open data, open science, open source"

@spetznatz, @danielhu & @cobismith were discussing whether age is a factor or whether it's about peoples' institutional positions and perspectives. @spetznatz noted that participation in open science skews young, but hopes that people will maintain enthusiasm for open science as they age.
@spetznatz said: "I think open science in unequivocally a good thing and is a natural reaction to new technology making old models irrelevant" He shared the example of closed publications.

! university of melbourne staff who might engage with content accessible through melbourne.figshare.com

Karen Mecoles

? researchers from UniMelb who have never heard of Figshare but might be interested in piloting it

@robrkerr

@engkerry

  • PhD data, which is part of the Bionics Institute, is stored within a hospital because it is clinical data.

^ researchers from UniMelb who have heard of Figshare but never used it but might be interested in piloting it

^ example:
@materialpaul (Formerly @paulmignone)

  • PhD funded by Defence Materials Technology Centre and thus intellectual property from research is not only student's decision to release for access.
  • Publications first require a discussion between various stakeholders (companies, government and academic institution) to review work and ensure no restricted information is presented in publication.
  • Cobi & Paul discussed this work, which is now in the public domain:
    http://www.aip.org.au/info/sites/default/files/cmm/2013/to2_corrected.pdf
  • Paul is maybe going to become part of the pilot group :)

^ example:
@hdashnow

"particularly for papers about code I think it's important to put a preprint up, because you need something out there right now if you want people to use your code" - @hdashnow

@cobismith & @hdashnow discussed the search function of melbourne.figshare.com, contrasting the search results for "dashnow" in the Melbourne version, which displays 1 result:
https://melbourne.figshare.com/articles/Ten_Simple_Rules_for_Writing_a_PLOS_Ten_Simple_Rules_Article/1219829
compared to 12 results: http://figshare.com/articles/search?q=dashnow

^ example:
@danielhu
"I don't feel that open science products and activities are yet valued by the national and international funding environment, and as a result I struggle to priotizise them"
"The most effective way to incentivise researchers to use tools like Figshare and other research dissemination technologies is by incoporating them into the metrics by which researchers are judged - and this means publications and grant funding".
"I can't do anything about changing the system if I don't got a job"

~ researchers from UniMelb who have heard of Figshare, tested it in the past, but not shared research objects yet for various reasons:

~ example:
in @isa_kiko's case it was because she was working with other peoples' data and she didn't feel like she was an author, and there was no option then to remove oneself as an author. @cobismith had a similar experience to this.

  • researchers from Unimelb who have heard of Figshare but choose not to use it for various reasons

-example:
"90% of what I do is interfacing with the workflow of my supervisor and he's not going to change" anonymous

& - researchers from Unimelb who already have been using Figshare individually, including:
&a: example: @drclimate
used https://www.mozillascience.org/code-as-a-research-object-a-new-project/ to have a snapshot of code repo at the time of submitting to a journal for publication. Motivated by access to an archive/ particular version, rather than DOI for citation or institutional data retention policy.

` - people in Unimelb staff community who don't perceive that the work they do is research

anonymous, @heydejan, @louwerff, Karen Mecoles, Participant Z, @gavinfry

@heydejan said: "I'll happily contribute if there's demand for my research objects - it depends on the scope of Figshare. I've thought of a creative commons license for blogging my assessments as a Masters student in Public Policy and Management, but wasn't sure of the university protocol and whether I owned that piece of work once submitted for assessment"

@louwerff raised these questions:
"what is the definition of a researcher?"
"do we want to encourage people to identify as researchers?"
"our role is to enable research rather than being a researcher per se"

Karen Mecoles said that the Figshare tshirts are really good. "I saw the Figshare tshirts at a conference recently, the eresearch 2015 conference. They had run out of tshirts at the conference but they sent me one and so I love Figshare!"

Karen Mecoles said "my work helps facilitate researchers to do their research. I don't make any research objects".

@gavinfry
"we do research inside our own environment, which facilitates research in a greater context"

@gavinfry

  • medical data - compliance with privacy act & IP issues are barriers to data sharing
    "deindentifying clinical research data can enable greater research collaboration"

Participant Z:
"we are the enablers"

Results: baseline Figshare awareness in self-selecting sample at #mozsprint

(ID default is Twitter handle - non-anonymous participants were asked to nominate a public ID. New participants were likely influenced by preceding participants' choice.)

heard of figshare?

  • @engkerry n
  • @spetznatz y
  • Karen Mecoles y
  • @danielhu y
  • @robrkerr n
  • @drclimate y
  • @paulmignone y
  • @heydejan y
  • @isa_kiko y
  • anonymous y
  • @hdashnow y
  • @louwerff n
  • anonymous y
  • @iniandra n
  • Participant Z n
  • @gavinfry y

total = 16
y = 11, n = 5

Knew of Melbourne piloting Figshare for institutions?

  • @spetznatz y
  • Karen Mecoles n
  • @danielhu y
  • @robrkerr n
  • @drclimate y
  • @paulmignone n
  • @heydejan y
  • @isa_kiko n
  • anonymous y
  • @hdashnow n
  • @louwerff n
  • anonymous y
  • @iniandra n
  • Participant Z n
  • @gavinfry y
  • @engkerry n

total = 16
y = 7, n = 9

Note: several people who said they'd heard of Figshare said they didn't know what it does, so bear in mind having heard of Figshare does not imply knowledge of what it does or experience with using it. This question could be better worded for clarity in future studies.

Results: participants specifically from #mozsprint willing to be contacted & potential research themes emerging for follow-up workshops

Bioinformatics?

h.dashnow@gmail.com
https://github.com/hdashnow

steve.androulakis@monash.edu

Data for sustainable development?

@iniandra
yismiraldi@student.unimelb.edu.au
https://github.com/yuandra

Negotiating data sharing?

mignone.p@unimelb.edu.au
http://www.chemeng.unimelb.edu.au/people/rhd.html

Medical imaging?

gfry@unimelb.edu.au
it.mdhs.unimelb.edu.au

Not yet explored themes

robrkerr@gmail.com
Computational Neuroscientist and aspiring Web Developer

Karen Mecoles
karen.mecoles@nectar.org.au
http://shakingthetreechoir.com/

isa.kiral@gmail.com

kerry.Halupka@gmail.com

Themes that would be useful to explore

Data in the arts - with VCA?

Data in law - with MLS?

Research: emerging issues

Citing records from databases

At #hackyhour on 2/67/15 Cobi spoke with Tim about the [Human Variome Project] (http://www.humanvariomeproject.org), which is an international non-governmental organisation that is working to ensure that all information on genetic variation and its effect on human health can be collected, curated, interpreted and shared freely and openly. At the moment, there are [databases relevant to the project listed] (http://grenada.lumc.nl/LSDB_list/lsdbs) of varying quality and levels of openness. Also some of the record objects are lists of other records, rather than records themselves. This is challenging for using and accessing data in meaningful ways. Tim shared this as an example of a well-structured dataset: http://chromium.lovd.nl/LOVD2/colon_cancer/home.php?select_db=MLH1 which is an example of a [gene/disease specific dataset] (http://www.humanvariomeproject.org/activities/gene-disease-specific-databases.html). We discussed a method of systematic documentation and analysis of human genetic variation in hemoglobinopathies using the microattribution approach (Giardine et al. 2011). We discussed Figshare as a platform for sharing as well as tools from centerforopenscience.org.

Structuring databases and projects in ways that support safe sharing

At #hackyhour on 2/67/15 Cobi and Tim discussed how databases can be structured in ways that support safe sharing, noting however the risks of possible reidentification from novel genetic variants from smaller deindentified genetic research datasets. Also present at #hackyhour was an anonymous aspiring systems biologist (Tim suggested we add 'citation needed' here) who shared thoughts on whether this was a database structure problem or a political problem. The three of us discussed issues including "circle of trust" in which people need to take a "white hat" approach to working with sensitive datasets. Safety depends on individual researchers taking responsibility - however how does this individual responsibility relate to responsibilities to support learning about diseases and training new researchers so that we can address global problems? We descended into discussions about worst case scenarios versus the public good. We discussed how "people are inherently terrible at understanding risks". However what if you have a child with an unknown life-threatening and rare disorder? In such cases families may want genetic information available far and wide so they can get a diagnosis.

Mediated deposits

As of June 2015 a function in Minerva and an item on the roadmap for Figshare @ UniMelb is "impersonation". This term rings alarm bells for some people and raises issues about informed consent. Demand for "impersonation" exists because of manual data entry work related to reporting to Australian research funders. Researchers may not find or make time to do this data entry work, so it gets done by administrative staff. Rather than framing this work as impersonation, framing it in a way that makes transparent who is doing that work and acknowledging them for it could have benefits in staff feeling more valued.

This could be improved by giving users access to a transparent log of who is adding information. This reflects practices in successful collaborative platforms such as Wikipedia (Reagle 2010). For example - say a professor logs on and discovers that someone (for the sake of the example called Dora) has done 5 hours of work doing data entry for them, the system should make this clear and support the professor realizing "oh - that thing that I thought was automated is actually Dora doing hours of work on my behalf!". Improved transparency about what processes are automated and what is manual labour might benefit perceptions of community and valuing others' contributions among staff.

In exploring alternative frames and practices for this, @dfflanders shared the Simple Web-service Offering Repository Deposit (SWORD) protocol. This protocol refers to this process as a mediated deposit and recommends using the term "on behalf of". Preserving the identity of both the depositing user and who they are depositing for is included in the protocol. Reframing impersonation as mediated deposits and making clear when people are depositing on behalf of others in line with the SWORD protocol would support interoperability and transparency.

Bibliography

Battarbee, K., & Koskinen, I. (2005). Co-experience: user experience as interaction. CoDesign, 1(1), 5-18.
Accessed online 15 June 2015 at: http://www-ist.massey.ac.nz/Plyons/Papers%20(by%20others)/HCI/User%20Experience%20Design/Battarbee%20Koskinen%202005%20Co-experience%20user%20experience%20as%20interaction%20Codesign%201%201%205-18.pdf

Cagan, J., & Vogel, C. M. (2002). Creating breakthrough products: Innovation from product planning to program approval. Ft Press.

Carberry, P. S. (2001). Are science rigour and industry relevance both achievable in participatory action research. From "Science and Technology: Delivering Results for Agriculture?". Edited by Barry Rowe, Danny Donaghy and Neville Mendham. Proceedings of the 10th Australian Agronomy Conference, January 2001, Hobart, Tasmania. Accessed online 11 June 2015 at: http://www.regional.org.au/au/asa/2001/plenary/5/carberry.htm

Chien, C. V. (2015). Exclusionary and Diffusionary Levers in Patent Law. Southern California Law Review, Forthcoming. Available at SSRN: http://ssrn.com/abstract=2624692

Dawson, E. (2012). Non-participation in public engagement with science: A study of four socio-economically disadvantaged, minority ethnic groups. Doctoral dissertation, King's College London

Dawson, E. (2014). Reframing social exclusion from science communication: moving away from ‘barriers’ towards a more complex perspective. Journal of Science Communication. Accessed online 11 June 2015 at: http://jcom.sissa.it/sites/default/files/documents/JCOM_1302_2014_C02.pdf

Fischer, F. (1999). Technological deliberation in a democratic society: the case for participatory inquiry. Science and Public Policy, 26(5), 294-302.

Giardine, B., Borg, J., Higgs, D. R., Peterson, K. R., Philipsen, S., Maglott, D., ... & Hardison, R. C. (2011). Systematic documentation and analysis of human genetic variation in hemoglobinopathies using the microattribution approach. Nature genetics, 43(4), 295-301. Accessed online 2/7/15 at: http://www.nature.com/ng/journal/v43/n4/full/ng.785.html

Grand, A., Wilkinson, C., Bultitude, K., & Winfield, A. F. (2012). Open Science A New “Trust Technology”?. Science Communication, 34(5), 679-689.

Heron, J., & Reason, P. (1997). A participatory inquiry paradigm. Qualitative inquiry, 3(3), 274-294.

Kindon, S., Pain, R., & Kesby, M. (Eds.). (2007). Participatory action research approaches and methods: Connecting people, participation and place. Routledge.

Koepsell, D. (2010). Back to basics: how technology and the open source movement can save science. Social Epistemology, 24(3), 181-190.

McTaggart, R. (1991). Principles for participatory action research. Adult Education Quarterly, 41(3), 168-187.

McTaggart, R. (1994). Participatory action research: Issues in theory and practice. Educational Action Research, 2(3), 313-337.

Muukkonen, H., Lakkala, M., & Hakkarainen, K. (2008). Technology-enhanced progressive inquiry in higher education. Encyclopedia of information science and technology IV, 3714-3720. Accessed online 17 June 2015 at: http://ebooks.narotama.ac.id/files/Encyclopedia%20of%20Information%20Science%20and%20Technology%20(2nd%20Edition)/Technology-Enhanced%20Progressive%20Inquiry%20in%20Higher%20Education.pdf

O'Neill, D. K., & Gomez, L. M. (1994). The Collaboratory Notebook: A Networked Knowledge-Building Environment for Project Learning. Accessed online 17 June 2015 at: http://files.eric.ed.gov/fulltext/ED388279.pdf

Piller, F., Ihl, C., & Steiner, F. (2010, January). Embedded toolkits for user co-design: A technology acceptance study of product adaptability in the usage stage. In System Sciences (HICSS), 2010 43rd Hawaii International Conference on (pp. 1-10). IEEE.

Ramachandran, D., Kam, M., Chiu, J., Canny, J., & Frankel, J. F. (2007, April). Social dynamics of early stage co-design in developing regions. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1087-1096). ACM. Accessed online 17 June 2015 at: http://bid.berkeley.edu/files/papers/chi07_ram_kam.pdf

Reagle, J. M. (2010). Good faith collaboration: The culture of Wikipedia. MIT Press. Accessed online 22 June 2015 at: http://iris.lib.neu.edu/cgi/viewcontent.cgi?article=1002&context=books

[Someone else is editing this]

You are editing this file