DISC Committee Meeting Minutes

From NumFOCUS Wiki
Jump to: navigation, search

Thursday, January 24, 2019 - 1:00-2:00pm Central

In Attendance:

  • Gina Helfrich
  • Reshama Shaikh
  • Samuel Brice
  • Leonie Mueck
  • Julia Meinwald

Not in Attendance:

  • Julie Hollek
  • Madicken Munk

What remains to be done between now and mid-March?

  1. Finalize “to complete” work on Moore grant
  2. Discuss potential priorities for new DISC work
    1. Finish CoC work with NF projects, events
      1. Affiliated Projects spreadsheet
      2. Sponsored Projects spreadsheet
      3. Include in newsletter to projects re: adopting the NF CoC
        1. If not, why?
      4. Individual outreach to projects that don’t respond
      5. Start with projects that are most out of compliance :)
      6. Make sure to include DISCOVER Cookbook reference with same communication
      7. https://numfocus.org/code-of-conduct#enforcement
    2. DISCOVER Cookbook but for OS project contributors
      1. Already have some lit review
      2. Next: identify potential metrics signifying levels of diversity & inclusion
      3. Develop project goals and see to what extent the lit review is compatible with them
    3. Focus in on specific area(s) of “diversity” (e.g. “women” and “engagement with S. America + Africa”) - ?
      1. One specific project about the global south (?)
        1. Include a narrative about why the focus is chosen
      2. Could keep us honest about whether we’re making a difference
        1. Has to go hand-in-hand with assessment/tracking plan
        2. Do we need to make this wait until the contributor-focused cookbook work is more complete? To know which metrics to focus on and measure against.
      3. Could we pick 3 or 4 dimensions to have on the list, even if we start small?
        1. How to execute?
        2. One larger focus and a few other smaller ones.
    1. Events idea: Topical event or conference track with how-to’s on debiasing data science workflows? (e.g. using biased/problematic NLP libraries)
      1. Table for further discussion
  1. Review how the current subcommittee structure is working/not working (incl. volunteer participation)
    1. Mixed experience so far with volunteers
    2. Made things easier when everyone was in the same timezone
    3. Helped to set working meetings
    4. More difficult to keep up with all of the subcommittees
      1. Subcommittee chairs can send monthly updates via email
  2. Review DISC Committee elections plan and make any needed updates
    1. Nominations open announcement early March
    2. Selected committee members announcement mid-April
    3. Members rolling off: (would like to stay on email distribution list)
      1. Julie Hollek
      2. Julia Meinwald
      3. Reshama Shaikh
    4. Board member to sit on DISC committee
      1. Jennifer Klay was a member, has rolled off NF Board
      2. Have asked the new NF Board if they want to send a representative

Review of work completed on Moore grant, ending March 15

  • Goals & status of goals for the Moore Foundation 2-year DISC grant:
    1. production of a scalable kit that can be disseminated to events to help promote best diversity and inclusion practices (including distribution plan)
      • STATUS: ALMOST COMPLETE
        • DISCOVER Cookbook is live and revised
      • TO COMPLETE:
        • Merge GH pages site back into NumFOCUS org
        • Make explicit/document the distribution plan
      • NEXT STEPS/NEW GRANT:
        • Finding ways to publicize the DISCOVER Cookbook to other communities
        • Develop tactics to help measure successful implementation of the Cookbook suggestions (+feedback mechanism)
          • Create checklists (believe this is in process)
          • Leonie to develop example feedback form (in TypeForm)
          • Can do a lot more on social media
            • Quiz/test? (use TypeForms) “How inclusive is your open source event?” Link at the end to the cookbook.
              • Could potentially collect anonymous data to see how successful people are
    2. assessment of current diversity programs at NumFOCUS
      • STATUS: COMPLETE
        • Post-Event Survey was developed & implemented
        • NF CoC and project CoCs were assessed
        • NF CoC was updated
      • NEXT STEPS/NEW GRANT:
        • Review & revise event post-survey questions
        • Finalize approach to data processing of results
        • Outreach to NF projects re: status of their individual CoC, possible improvements
    3. identify best diversity practices and challenges, plus suggestions for new programs
      • STATUS: COMPLETE
        • Best practices identified within DISCOVER Cookbook
        • Challenge: we need a “DISCOVER Cookbook for Contributing”
          • Projects have articulated need for guidance on how to support inclusive and diverse contributor communities
        • New programs suggested & launched so far:
          • DISC Outreach Events
            • DISC Unconference
            • Open Data Science Inclusion Sprint
      • NEXT STEPS/NEW GRANT:
        • Create proposal/outline for “DISCOVER Cookbook for Contributing”
          • Assessment has been collecting research on potential metrics for this project
        • Invent snappy name/acronym
        • Define NF DISC events strategy
    4. creation of initiatives including programs to increase the diversity of project contributors
      • STATUS: COMPLETE
        • Initiatives created:
          • DISCOVER Cookbook
          • DISC Outreach Events
            • DISC Unconference
            • Open Data Science Inclusion Sprint
          • Diversity session included at NF Summit
      • NEXT STEPS/NEW GRANT:
        • Generate “map” of potential pipelines into project contributing
        • Assess map for opportunities to create intervention initiatives

Follow-up actions on Next Steps:

  • Finding ways to publicize the DISCOVER Cookbook to other communities
  • Develop tactics to help measure successful implementation of the Cookbook suggestions
    1. Create checklists (believe this is in process)
  • Review & revise event post-survey questions
  • Finalize approach to data processing of results
  • Outreach to NF projects re: status of their individual CoC, possible improvements
  • Create proposal/outline for “DISCOVER Cookbook for Contributing”
  • Invent snappy name/acronym
  • Define NF DISC events strategy
  • Generate “map” of potential pipelines into project contributing
  • Assess map for opportunities to create intervention initiatives

Gina to send whenisgood re: meeting in late March/early April to select new members.

Also add Katrina Riehl as BoD member. (add to email list)

Wednesday, October 10, 2018 - 1:00-2:00pm Central

In Attendance:

  • Gina Helfrich
  • Reshama Shaikh
  • Madicken Munk
  • Julia Meinwald
  • Samuel Brice

Not in Attendance:

  • Julie Hollek
  • Leonie Mueck

Discussion Topics:

Subcommittee check-ins/updates

  1. Assessment (Madicken)
    • Updates & Goals
      1. Assessment of Previous Events
        1. Use Google Analytics? (Privacy concerns)
          1. For what? (TBD)
        2. Next meeting today
        3. Kasia will replicate Reshama’s analysis
        4. Mo & Carlos are cleaning the data
        5. Assess whether to update the form
        6. Port the survey to Typeform
          1. How to link to NumFOCUS typeform? Gina can add users to the NF account; port from there.
      2. DISCOVER Cookbook
        1. Been moving existing repo to a webpage that’s more accessible (Jekyll) https://discover-cookbook.github.io/
        2. keywords/taxonomy of content
        3. Strategy for promoting it
          1. Last meeting talked about tweeting pieces of the actual COOKBOOK
      3. Lit Review
        1. Group citation manager
        2. Overall summary doc to start taking notes in
        3. Strategically divvying up the work
          1. Sorting how to do the lit review (each person has a sub-topic?)
            1. How to delegate papers
            2. How to sort
  2. Community Outreach (Sam & Julia)
    • Updates & Goals
      1. Friday Oct 26 Open Data Science Inclusion Sprints at Two Sigma
        1. GitHub page
        2. 8 sign-ups so far
        3. Sam reached out to Dev/Color, WSJ, Two Sigma ERG
        4. Need to make sure people know about the 100% code (DISC)
        5. Need to do more targeted outreach this week
        6. Student outreach
  3. Code of Conduct (Gina)
    • Updates
      1. CoC session at NumFOCUS Summit
      2. Revised code is here, blog about it is here.
      3. Email went out this week reminding/notifying projects.
      4. Just got an inquiry email for help from ArviZ library.
      5. Subcommittee meetings on hold until next steps of work can be identified.  (thoughts?)
    • Goals
      1. Next steps:
        1. Outreach message to PyData Meetup organizers notifying them of the revised code and reporting procedures.
        2. Outreach to PyData conference organizers notifying them of the revised code and reporting procedures.
        3. Individual outreach to projects about their own codes.
          1. Would subcommittee members want to help with this?
            1. Cc Gina, make sure subject line is clear
            2. Template email to get started
        4. Hire consultant to provide training to NumFOCUS staff & board, PyData organizers, project reps on promulgating code, responding to reports.
        5. Draft response procedures/enforcement manual.

DISC Unconference 2019

  • Spoke with Chris at Moore Foundation
    1. Encouraged us to try to collaborate to host versions of it at other major scientific computing conferences, such as UseR in Toulouse
    2. Academic professional societies, e.g. American Statistical Association
    3. RuST community (sp/caps?) -- been doing a lot of inclusion work
  • Questions, need help/input on anything?

Wednesday, August 1, 2018 - 2:30-3:30pm Central

In Attendance:

  • Gina Helfrich
  • Leonie Mueck
  • Jennifer Klay
  • Julia Meinwald

Not in Attendance:

  • Reshama Shaikh
  • Samuel Brice
  • Madicken Munk
  • Julie Hollek

Discussion Topics:

  1. Ford Foundation Digital Infrastructure Grant Proposal - not accepted :( (only about 5% of proposals were advanced)
  2. NumFOCUS Board Elections
  3. Subcommittee check-ins/updates
    1. Assessment (Madicken)
      1. Volunteers are in Slack now; has been a bit slow over summer
      2. Big projects are:
        1. making the DISCOVER Cookbook more presentable and accessible (Leonie +2 volunteers w/web background). They are developing a taxonomy for the recommendations.
        2. Literature search. Can we identify good metrics for diversity and inclusion in open source? Not just demographics, but culture/atmosphere.
        3. Assessment of event surveys. Replicating the work Reshama did for PyData Seattle 2017.
    2. Community Outreach (Sam & Julia)
      1. Co-locate a hackathon with PyData NYC either Oct 17 or 18; theme is Machine Learning In Jupyter. Jim (of NF) and Robin (of NYTimes) supporting. Hoping for up to 60 participants. Looking at doing additional workshops, e.g. how to contribute.
    3. Code of Conduct (Gina)
      1. Put together a spreadsheet audit of NumFOCUS project Codes of Conduct, follow-up is happening or will happen soon
      2. Gina has incorporated comments from the subcommittee into a revised version of the CoC; still outstanding is the Enforcement guidelines/manual section
      3. Hope to complete this process by the Summit in late September
  4. Questions, need help/input on anything?
    1. If you have any updates on your work, please send them to Gina because she will be making a presentation about DISC at the Summit.

Next Full Committee Meeting:

  • Gina to send around a new poll for best day/time. Please refer to your fall schedule.
  • Goal to meet next in early October, after NumFOCUS Summit.

NumFOCUS DISC Committee Meeting - 06.25.18

In Attendance:

  • Gina Helfrich - gina@numfocus.org
  • Leonie Mueck - lmueck@plos.org
  • Samuel Brice - brices@gmail.com
  • Madicken Munk - madicken.munk@gmail.com
  • Reshama Shaikh - reshama.shaikh@gmail.com
  • Julia Meinwald - Julia.Meinwald@twosigma.com

Not in Attendance:

  • Julie Hollek - julie.hollek@gmail.com
  • Jennifer Klay - jlklay@gmail.com

Discussion Topics:

  1. Ford Foundation Digital Infrastructure Grant Proposal (Gina & Madicken)
    1. Gina & Madicken submitted a proposal to identify metrics that indicate success in attracting & onboarding new contributors. Will know in about a month if accepted for full proposal.
  2. Subcommittee check-ins/updates
    1. Assessment (Madicken)
      1. Met last week; not all volunteers were able to attend. Webpage for DISCOVER Cookbook & incorporating info from other projects on it. They’ve been messaging volunteers to get people assigned on the different tasks. Have a “TL;DR” of the cookbook so people don’t have to read the whole thing.
      2. Literature review to see if people have measured open source onboarding tools to see if they’re inclusive. Coordinate w/volunteers and choose transparent way to collect the literature. (Citation manager?)
      3. Looking at extant assessment data from surveys. Reshama walked through how the Seattle analysis was done. Volunteer will try to replicate that on other data sets.
      4. Working to create autonomy within the different projects.
    2. Community Outreach (Sam & Julia)
      1. Got one volunteer; will be reaching out to them.
      2. In touch with DevColor group; good responses from about 5 ppl. Thinking about how to format a hackathon, get their help to get a bigger group.
      3. More updates coming on the email threads.
    3. Code of Conduct (Gina)
      1. Meeting tomorrow; have two volunteers.
      2. Ralf Gommer (NF Board Member) sent some food for thought via email.
      3. NumFOCUS looking forward to recommendations so we can use them at the Summit in September.
  3. Questions, need help/input on anything?

Next Full Committee Meeting:

  • Wed., August 1st from 2:30 – 3:30pm

NumFOCUS DISC Committee Meeting - 05.18.18

Attendees Present:

  • Leonie Mueck
  • Sam Brice
  • Madicken Munk
  • Reshama Shaikh

Not in Attendance:

  • Julie Hollek
  • Julia Meinwald
  • Jennifer Klay

Discussion Topics:

1) Housekeeping

a) Proposed summer meetings: (ok with Julie, Julia, Jennifer?)

  • Monday, June 25th 1:15-2:15pm Central
  • Wednesday, August 1st, 2:30-3:30pm Central
  • We will meet bi-monthly to accommodate monthly subcommittee meetings.
  • FYI Poll result - summer best meeting times:
    • Mondays between 1:00-2:30pm Central
    • Wednesdays between 2:00-4:00pm Central
      • Secondarily:
        • Thursdays between 1:00-1:45pm Central
        • Fridays between 2:00-2:45pm Central

b) Bios & photos still needed from:

  • Julie H.
  • Julia M.
  • Jennifer K. (use website one?)
2) Finalize immediate goals of Subcommittees

Subcommittee

Description of Goals

Code of Conduct

Chair: Gina Helfrich

Julie Hollek

Reshama Shaikh

  • Review and discuss appropriate updates to the current NumFOCUS Code of Conduct.
  • Consider and plan for the many ways that NumFOCUS convenes communities that will need to be educated about and governed by the CoC.
  • Develop a proposal for CoC reporting procedures in line with current best practice.
Assessment

Co-chairs:

Madicken Munk

& Julie Hollek

Reshama Shaikh

Jennifer Klay

Leonie Mueck

  • Promote, update, and improve usability of the DISCOVER Cookbook.
  • Make a GitHub pages fully working website for the DISCOVER Cookbook.
  • Collate and analyze results of event surveys administered at PyData events.
  • Develop KPIs for assessment of NumFOCUS diversity & inclusion programs — in particular, the DISC program and the PyData diversity programs.
  • Come up with metrics for measuring a culture of inclusion; create recommendations for various NumFOCUS projects in the form of a cookbook.
Community Outreach

Co-Chairs:

Samuel Brice

& Julia Meinwald

Leonie Mueck

  • Develop a plan for a second session of the DISC Unconference in 2019. (May involve assessment of the first DISCunconf.)
  • Develop and implement plans for strategic events targeted at improving inclusion and diversity in scientific computing, for example through distributed but simultaneous hackathons.
  • Suggest strategies for attracting and training more diverse contributors to NumFOCUS open source projects.

CoC:

Madicken suggests an anonymous form for feedback about the process, after the resolution. (“How did we do?” evaluation.)  Are we maintaining an atmosphere of inclusion with regard to the CoC?

Assessment:

Gina to send Madicken info re: PyData surveys.

Digital Infrastructure Research RFP

Community Outreach:

Reshama’s blog:  Resources for Women in Data Science

Stack Overflow’s blog on being more inclusive

3) Finalize plan for volunteer participation in Subcommittee Work —

Send call for volunteers information to disc-committee-at-numfocus.org no later than

May 24.

A) Verbiage for call for volunteers for each subcommittee:

DISC Volunteers devote 3-4 hours of their time each month to subcommittee tasks.

We are looking for 3-5 volunteers to…  

Tasks will include…

It is helpful to have experience with…

Skills needed are…

Meetings happen on…

Respond with your interest no later than...

B) Subcommittee convenes meetings once a month at least.

  • Poll result - summer best meeting times:
    • Mondays between 1:00-2:30pm Central
    • Wednesdays between 2:00-4:00pm Central
      • Secondarily:
      • Thursdays between 1:00-1:45pm Central
      • Fridays between 2:00-2:45pm Central

Leonie & Madicken to put together Assessment summary.

C) Gina will send names & contact info of those who have already indicated a willingness to volunteer to the chairs of each subcommittee.

  • Please follow up with them by providing the details above in “B.”

D) Gina will send a new call for volunteers to the DISC email list.

E) Please take notes/minutes at Subcommittee meetings and share them in the DISC Google Drive folder!

NumFOCUS DISC Committee Meeting - 04.19.18

Attendees Present:

  • Gina Helfrich
  • Julie Hollek
  • Jennifer Klay
  • Samuel Brice
  • Julia Meinwald
  • Leonie Mueck
  • Madicken Munk
  • Reshama Shaikh

Discussion Topics:

1) Committee Member Introductions

  • 2018-2020 term:
    • Leonie Mueck
    • Samuel Brice
    • Madicken Munk
  • 2017-2019 term:
    • Julie Hollek
    • Jennifer Klay (NumFOCUS Board Member)
    • Julia Meinwald
    • Reshama Shaikh

Gina will send an email to everyone requesting your locations & job titles so that we can publish the full roster on the NumFOCUS website.

2) Review and Plan how to organize Subcommittee Work

  1. Existing subcommittees: Assessment, Code of Conduct
    • We have volunteers! Folks who applied for the Committee but weren't selected have said they would like to help:
      • Assessment: 3 volunteers so far
      • CoC: 2 volunteers so far
  2. Moore Grant remainder work - For the new members, Gina went over the goals of the Moore Foundation grant to the DISC program and the status of each:
    • Production of a scalable kit that can be disseminated to events to help promote best diversity and inclusion practices (including distribution plan)
      • The DISCOVER Cookbook has been created; now we are looking for events to implement it and assess their results:
        1. Jennifer explained that the NumFOCUS Board is now requiring recipients of grant funds from NF to use the cookbook and report back; this will be incorporated into the application process.
        2. We could still use more outreach to events that aren't receiving NF grant funds to ask if they're interested and willing to implement & assess the recommendations.
    • Assessment of current diversity programs at NumFOCUS
      • Post-event survey was developed & implemented
      • More analysis and refinement of survey is needed; e.g. it is too long
      • Additional program assessments beyond the survey may be warranted
    • Identify best diversity practices and challenges, plus suggestions for new programs
      • Many best practices and challenges identified as part of generating post-event survey & DISCOVER Cookbook
      • New program suggested & launched so far: DISC Unconference
      • Other new programs? Future of Unconference?
        • This needs to be explored. Given the preparation necessary for a successful DISC Unconference and the other event-related commitments NumFOCUS has this year, the next version of the DISC Unconference will likely need to be scheduled for 2019.
    • Creation of initiatives including programs to increase the diversity of project contributors
      • More work is needed to assess strategies for more diverse contributors
  3. The Committee agreed to formalize the following Subcommittees for 2018:
    • Code of Conduct - Gina (chair)
      • Julie will provide input; Reshama can pitch in
    • Assessment - Julie and Madicken (co-chairs)
      • Leoni to pitch in; good at project managing. Reshama can pitch in, as well.
        • DISCOVER Cookbook - Jennifer to head up this project as part of the Assessment subcommittee.
          • Ensure it’s being utilized and analyzed.
    • Community Outreach - Sam and Julia (co-chairs)
      • Programs & Events focus
        • Hackathons, targeted tutorials
        • Outreach! Grassroots efforts that make it easy to interact with DISC & NumFOCUS.
  4. Because previously "subcommittees" were just made up of DISC Committee members, we need to think about how to organize the work in order to best incorporate volunteers and maximize effectiveness & efficiency.
      • Subcommittees should convene at least once a month. The Committee agreed it is generally most effective to convene "real" meetings (meaning call or video conference).
      • There was discussion of whether two meetings per month is more than the Committee members can reasonably handle; it was floated to hold DISC Committee meetings to 30 minutes once a month, but ultimately a different option was preferred:
      • In order to cut down on excessive meetings, the DISC Committee will meet as a full group only every other month, on the idea that this will provide two months' worth of subcommittee work to discuss and report on at full Committee meetings. Once per quarter seemed too infrequent.

For scheduling future DISC Committee meetings:

Leonie is based in London; the rest are in the U.S. (When it's 12:00pm in Austin it's 6:00pm in London.) Leonie is available until 11:00pm London time.

We will keep the existing May meeting on the calendar (Friday May 18, 1:30pm Central) and send a poll to collect the best days and times for June/July and the rest of the summer. We will re-visit availability again when the calendar changes for the start of fall term. At the May meeting, we will reconvene to discuss in more detail how to manage subcommittee work, immediate goals/tasks for each subcommittee, the recruitment of volunteers, etc.

NumFOCUS DISC Committee Meeting - 03.06.18

Attendees:

  • Gina Helfrich
  • Reshama Shaikh
  • Jennifer Klay
  • Julie Hollek
  • Alex Companioni

Not Present:

  • Manuel Rivas
  • Julia Meinwald
  • Erica Moszkowski

Agenda

1) DISC Elections plan (15 min.) - Plan is here

Timeline:

Wed. March 7th (tomorrow) — publicize nominations

Sun. March 25th — nominations close

Thurs. April 5th — applications due (this date was moved up from the original proposal in order to ensure the committee has two weekends to review applications)

Mon. April 16th — notifications sent

Thurs. April 19th — new committee term begins

Nominations announcement

2) Sub-Committee work; chairs needed; can solicit volunteers from the community and Committee applicants who aren't elected (30 min.)

Gina asks everyone to consider where they'd like to put their energies and let her know at the April meeting which sub-committee they will work on/chair.

  1. Elections (solicit nominations, craft application and voting procedure)
  2. Assessment (including assessment of the DISCOVER Cookbook, conference/PyData survey, and Unconference; identify other assessment work needed to advance Moore Grant goals)
    1. Update from Jennifer re: outreach to other conferences to use the cookbook/funds available via NumFOCUS
      1. This can be part of all NumFOCUS grant-making to conferences.
      2. Ask for one-page report on efforts they made using the cookbook (perhaps)
      3. Start interfacing directly with the conference committees NumFOCUS already supports (e.g. PyData)
    2. Jennifer volunteers to serve on assessment sub-committee
    3. Julie volunteers to work on metrics/analytics — discussion of possibly making this its own sub-committee
  3. NumFOCUS Code of Conduct (develop a revised CoC based on the work/research done for the Cookbook)
    1. Reshama volunteers to work on CoC
  1. Other business
    1. On the Committee Application, ask "If not elected, would you be interested in volunteering on a sub-committee?"
    2. It would be useful to create a list of conferences to promote the Cookbook to
    3. Another task for someone (a volunteer?) could be to create a better GitHub pages version of the Cookbook and a web app to collect reports on using the Cookbook
    4. Create a working list of volunteer tasks that can be publicized
    5. Gina to send short paragraph re: GSoC for publicity to UR groups — historically we have struggled to have much diversity in our GSoC cohort of students

NumFOCUS DISC Committee Meeting - 02.09.18

Attendees:

  • Gina Helfrich
  • Reshama Shaikh
  • Jennifer Klay
  • Julie Hollek
  • Julia Meinwald

Not Present:

  • Manuel Rivas
  • Alex Companioni
  • Erica Moszkowski

Agenda

  1. DISC Committee Structure Proposal (20 min.)
  2. Elections Sub-Committee (10 min.)
  3. Review of Moore Grant (30 min.)

1. DISC Committee Structure Proposal

  • Proposed Structure
    • Intended to provide clear structure but not to be overly rigid or formal
    • Additional volunteers can be welcomed to participate via sub-committees
  • Elections would take place in March 2018
  • Current terms would end at the April meeting, which is also when new members would join and their terms would start.
    • Agreed that a joint meeting seems fine; it can be changed if it doesn't work well.
  • In order to prevent whole-sale turnover of the committee every two years, half of the current committee should continue a second year and the other half should conclude their terms in April.
    • Continuing:
      • Julie Hollek
      • Jennifer Klay
      • Julia Meinwald
      • Reshama Shaikh
    • Gina to ask Alex, Manuel, and Erica to conclude their terms in April.

2. Elections Sub-Committee

  • 2-3 people (Julia M. volunteered)
    • Julie H. will help publicize
    • Jennifer K. will help craft application
  • Publicize to help ensure we receive a nice pool of nominees
    • Definite bias towards U.S.-based folks, due to need for synchronous monthly meetings; take a look at the applications and try to limit to 2 continents
  • Craft the application (what the committee is, why do you want to join)
    • Give some idea of the types of tasks people would be engaged in and say that it's open to ideas from the committee about what we will actually do. Give folks a chance to gauge what we've done and what they're signing up for, do they have some ideas they'd like to bring to the committee?
  • Establish voting procedure

3. Review of Moore Grant

  • Goals & status of goals for the Moore Foundation 2-year DISC grant:
    1. production of a scalable kit that can be disseminated to events to help promote best diversity and inclusion practices (including distribution plan)
      • STATUS:
        • Initial draft of kit is posted for community use & input
    2. assessment of current diversity programs at NumFOCUS
      • STATUS:  
        • Survey was developed & implemented
        • More analysis and refinement of survey is needed
        • Additional assessments beyond the survey may be warranted
    3. identify best diversity practices and challenges, plus suggestions for new programs
      • STATUS:
        • Best practices and challenges identified as part of generating survey & kit
        • New program suggested & launched so far: DISC Unconference
        • Other new programs? Future of Unconference?
    4. creation of initiatives including programs to increase the diversity of project contributors
      • STATUS:
        • DISC Unconference
        • More work is needed to assess strategies for more diverse contributors
  • Review Roadmap: https://wiki.numfocus.org/DISC_Moore_Grant_Roadmap
  • Review of DISCOVER Cookbook launch, next steps
    1. Identify a few existing conferences where we have contacts and ask them to use it, give us feedback
    2. Incentive? NumFOCUS resources in exchange for piloting it? $ for implementation in exchange for assessment data (survey and “user research” questions)
      • Jennifer to propose at NF Board meeting
    3. https://github.com/WiMLDS/conferences
  • Need to reconvene Survey/Assessment sub-committee for further work
  • Note: possible sub-committee for NumFOCUS Code of Conduct improvement

NumFOCUS DISC Committee Meeting - 12.12.17

Attendees:

  • Gina Helfrich
  • Reshama Shaikh
  • Jennifer Klay
  • Julie Hollek

Not Present:

  • Kari Jordan
  • Manuel Rivas
  • Alex Companioni
  • Erica Moszkowski
  • Julia Meinwald

Agenda

1. Report on recent activities

  • DISC Unconference 2017 at PyData NYC (Nov 29-30)
    • https://github.com/numfocus/DISC-unconf-17

2. Moore Grant updates

  • Best Practices for Conferences & Events Kit Roadmap
    • Public version of the kit will be up on GitHub by January for community input and feedback. Made great progress at the Unconference.

3. DISC Committee structure discussion

  • Term Limits Proposal
    • Term = 2 years, renewable (current terms would end in March)
    • Maximum terms served? Or up to the individual?
      • Mechanism for stepping down from the committee.
      • Reshama: One goal for 2018 is to get people beating down the doors to be on the DISC committee. 
      • Gina: Invite current committee to step down after 1-year so we can have alternating years for 2-year terms. 
      • Jennifer: yes, also gives folks an out who have realized they're actually too busy.
      • Reshama: Place for legacy committee members -- stay on distribution list? 
      • Gina: What do we mean by "distribution list"? Maybe a brief monthly newsletter for DISC.
  • Self-Electing Committee
    • Solicit nominations from the community
    • Election sub-committee?
    • Would there be a max number of committee members?
      • Jennifer: Max 7 Committee Members. Ideal to get an odd number for voting purposes. If people are banging down the door to be on the committee, then institute term limits. Suggested limit (not mandatory) to encourage turnover and fresh ideas. One year feels too short to be effective. 2-year terms.
      • Reshama: Short application: what the committee is, why do you want to join? For the future, raise the bar and ask why you want to participate.
      • Jennifer: With that solicitation, give some idea of the types of tasks people would be engaged in and say that it's open to ideas from the committee about what we will actually do. Give folks a chance to gauge what we've done and what they're signing up for, do they have some ideas they'd like to bring to the committee.
  • Committee Member Expectations
    • # approximate volunteer hours
    • Feedback on committee work & experiences to date
    • Would be a good idea to schedule monthly meetings many months in advance
      • Jennifer: It's been only a few hours a month. More time working on the Events doc. Averaged over 7-8 months, 15 hours total including meeting times. 5 hours a month—one meeting, plus research work, reading, tasks, etc. "That sounds like an appropriate amount and also too little."
      • Reshama: An hour a week on average.
      • Gina: Minimum 4-5 hours per month, definitely opportunity for more if you have the time and inclination.
      • Reshama: Easier to make progress when you have smaller sub-committees of 3 or 4. (At least one sub-committee per member? Also chairs for each sub-committee.)
      • Gina: I will re-survey the Committee in January about availability through June and set 6 months' worth of dates based on that survey.

4. Future of DISC Program

  • Proposal for overall direction of the program: Centralizing resources, increasing discoverability, connecting people to one another for knowledge sharing, taking a position on best practices
  • Your vision
  • Strategic planning/goal-setting process?
    • Jennifer: Have an org or project come to us and say "we want to do better, please help" -- we can "audit" their existing practices and provide suggestions for what to implement. Connect them to the right resources or provide what we have available.
    • Julie: Linking and uniting other orgs instead of spinning up our own efforts. Discovery mechanism is challenging for all the various D&I groups. Would be valuable to do calendar for PyLadies, SheTalksData, etc. Make it easier for those orgs to talk to each other. Centralized publicity? Doing that work would help us identify minoritized groups that aren't getting enough attention.
    • Reshama: Might take another year to cement DISC; hopefully will get to the point where people go to the repo or the project lead. Would be nice to get to the point that it feels owned by the community.

5. Other business

  • Feedback from the community
    • Details send by email to the committee.
  • Kari Jordan stepping down from the Committee due to overload of commitments

NumFOCUS DISC Committee Meeting - 11.15.17

Attendees:

  • Gina Helfrich
  • Julia Meinwald
  • Reshama Shaikh

Not Present:

  • Kari Jordan
  • Julie Hollek
  • Manuel Rivas
  • Alex Companioni
  • Jennifer Klay
  • Erica Moszkowski

Agenda

1. Updates

  • Resuming monthly meetings — Gina will ensure a new calendar invite goes out each month with at least 4 weeks notice, alternating Tuesdays & Wednesdays
  • Tobi Bosede has stepped down from the committee for the time being
  • Collaborative tools: wiki, etherpad also now available (thank you to Christie Koehler)
  • Carly Strasser from Moore Foundation was very pleased with my report to her about our activities and progress since March


2. Report on recent activities

  • Report from Intro to Data Science Women’s Coding Workshop in SF
  • NumFOCUS Summit discussion about diversity & inclusion
  • Gina to present on DISC Survey collection challenges at PyData NYC
  • DISC Survey — meeting Friday 11/17 to review with Moore Foundation social scientist
  • DISC Unconference 2017 at PyData NYC (Nov 29-30)

44 participants plus Gina, Reshama, Julia

80% identify as underrepresented in scientific computing

Fairly even split between new to open source and experienced open source contributors

Discussion active on Discourse

Unconference projects likely to include continued work on Conference Best Practices document

3. Best Practices for Conferences & Events Kit Roadmap (reminder)

4. Other business

Workshops for underprivileged communities (Manuel) — scope and details? — tabled until Manuel can join the meeting

Reshama asked what is the timeline of the Moore Grant? (It is a 2-year grant and finishes up in Spring 2019.)

It would be good to talk about "term limits" for the DISC Committee at a future meeting, so that people don't have to serve indefinitely and other folks can raise a hand to be involved. (e.g. one's official term is up after a year with the option to renew annually)


NumFOCUS DISC Committee Meeting - 08.30.17

Attendees:

  • Gina Helfrich
  • Jennifer Klay
  • Kari Jordan
  • Julie Hollek
  • Tobi Bosede

Not Present:

  • Alex Companioni
  • Erica Moszkowski
  • Manuel Rivas
  • Julia Meinwald
  • Reshama Shaikh

Agenda

1. Update on Intro to Data Science Women’s Coding Workshop in SF (Gina) Sept. 30th 9:30am-5:30pm @ USF Science Center


Lead instructor confirmed, TA's confirmed, catering confirmed, will make Eventbrite registration live and work with Kali of Hack the Patriarchy to recruit participants (undergraduate women & non-binary folk) Hope to hear follow-up from Manuel re: coding workshop for under-privileged communities in Miami.


2. Update on Conference Best Practices document (Jennifer)


Working document is in Drive; looked at research and reviewed collected resources, made notes of what seems to be best practice. Gina will take draft document with Reshama & Jennifer's notes (and her own) and work it up into a first draft of a full document for sharing. Once the committee signs off on it, we will upload it to a GitHub repo and solicit the community to make use of it, fork it, submit PRs to improve it, Issues for any problems, etc. Moore will be happy to have this.
Julie provided the below docs on Open Review:
SciPy 2017 Code of Conduct (updated from 2016)
Guidelines for reviewers in double open review systems


3. Update on PyData NYC DISC Unconference (Gina)

  • Dates: Nov. 29-30
  • Application Deadline: Oct. 8
  • Notification Date: Oct. 16
  • Thoughts on asking for demographic information to ensure diverse participants?


Tobi: Ask for evidence they are passionate about D&I; what their experience has been with scientific computing & level of diversity & inclusion; personal anecdotes. Push to ensure we have allies in the room. It's very important to make sure that targeted communities aren't the only ones doing the work and participating. Example questions:

"Please give examples of how you’ve championed D&I in the scientific computing community."

"What have you noticed about Diversity & Inclusion in scientific computing?"

"Do you identify with a group that is underrepresented in scientific computing? Please describe."


NumFOCUS DISC Committee Meeting - 07.11.17

Attendees:
Gina Helfrich
Jennifer Klay
Tobi Bosede
Reshama Shaikh
Erica Moszkowski

Not Present:
Manuel Rivas
Julia Meinwald
Kari Jordan
Alex Companioni

Agenda

  1. Survey update


a) Confidentiality update
b) Final checks

2. Other Updates

a) Workshops for underprivileged communities (Manuel) — Manuel & Kari & Gina met earlier to brainstorm; Manuel will send proposal outlining scope and details
b) SciPy diversity efforts (Gina) — Gina served on SciPy Diversity Committee; Julie Hollek presentation on their efforts
c) Undergraduate women intro to data science workshops (Gina) — Looking for venue for Sept 30th in SF; grant money is from OSDC
d) JuliaCon (Erica)

3. Next project: Best practices for organizing committees

  • a) Develop sketch of conference tasks and policies relevant to diversity & inclusion (Can also pull from assembled resources document.)
    What is the low-hanging fruit? Ask conference organizers in our network: What have they done? What did they find challenging? What had biggest impact? Who would like to help lead this project?
  • Reshma
  • Jennifer

Will also ask those not present.

SKETCH OF AREAS TO DRILL DOWN ON:

Org. Committee make-up

Code of Conduct

  • Reporting procedures
  • Response to violations

Outreach

  • Advocates/ambassadors for outreach — encouraging people to apply for the CFP, to apply for the diversity scholarships, to attend, to give keynotes, etc.
  • Community Org. partnerships

Speakers

  • Invited speakers
  • Selected speakers
  • CFP instructions
  • CFP review (“blind” vs “affirmative action”)

Services available on-site

  • Childcare
  • Accessibility
  • Bathrooms

Diversity Programming or Meet-Ups during the conference

  • BOFs
  • Meet & Greet

Registration process

Scholarships

  • Registration
  • Travel
  • Where to get funding?

NumFOCUS DISC Committee Meeting - 06.14.17

Attendees:
Gina Helfrich
Leah Silen
Kari Jordan
Alex Companioni
Tobi Bosede
Reshama Shaikh

Not Present:
Erica Moszkowski
Manuel Rivas
Julia Meinwald
Jennifer Klay

The focus for today's meeting was to review the D&I survey questions that Alex, Erica, and Gina drafted. We will discuss any specific questions that the committee wants to address, as well as determine an appropriate strategy for implementation of the questions into a survey.

Alex: Few things we tried to emphasize in the questions:

  • Better understanding to what degree we’re bringing in a diverse set of attendees
  • Diversity takes a lot of dimensions (demographics, experience, industry, toolsets, etc.)
  • His personal preference: tried to limit number of free-form answers

Gina: Can get good info from qualitative questions, although it's more time-consuming.

Alex: Can look at qualitative answers in light of specific cross-sections of answers to quantitative/demographic questions.

Kari: Appreciates self-identification approach to gender, ethnicity questions. Add a box that says “please write-in what you mean by that.” (Basically a both-and approach.) There may be groups who self-identify as underrepresented who we don’t even know about.

Reshama: Do we have “thank you for participating” language?

Gina: No, but we should draft that language. I can take a first crack and disseminate it for comments.

Kari: I can look at it once circulated. From Data Carpentry standpoint, people ask “why are you asking these questions? We should be past this racial/gender stuff.” Explain what the goal is and people will respond.

Gina: For a later meeting: discuss question of Review Committees and how to handle review process (removing identifying information vs. being pro-active in selecting for diversity). Questions to consider including in CFP: “Is this your first time submitting a CFP? (or is it: Is this your first time submitting to this conference?)” -- making explicit that this information won't be part of the decision-making process. "Would you like to receive feedback on why your proposal was not accepted?" for first-time submitters who were rejected (rather than: we've received numerous submissions, and can only pick a limited number).

Reshama: Should we ask country, nationality -- what if people interpret the “underrepresented” question differently? [Country of residence question was added.]

Tobi: I think the question is reasonably-posed as is; we can sit on it.

Gina: Should we keep or separate demographics & inclusiveness? Concern re: anonymity being compromised.

Kari: As part of introductory language, let them know what the info will be used for, that it’s optional, that they can skip any question they don’t feel comfortable answering.

Alex: I think that’s a great way to let people take the option to decide how much or little personal info to provide.

Reshama: We’ll be clear at the beginning whether survey is anonymous or not.

To discuss at next meeting: Sharing questions to other conferences; open-sourcing the survey and sharing results; concerns re: conveying how widely this data will be shared and with whom, anonymized to what extent.


NumFOCUS DISC Committee Meeting - 04.26.17

Attendees:
Gina Helfrich (NumFOCUS Communications Director; DISC Program Mgr.)
Leah Silen
Alex Companioni (Data Scientist at IHeartRadio; in Miami; Cuban heritage)
Tobi Bosede (Grad student at Johns Hopkins in Applied Math & Statistics; moving back to NY to be a Machine Learning engineer at Capital One; was active PyLadies NYC organizer; volunteered for Black Girls Code)
Reshama Shaikh (Data Scientist @ Metis; organizes NY WiMLDS)
Erica Moszkowski (Research Analyst, Fed. Bank of NY; Economics PhD in fall @ Harvard Business School, Diversity Chair @JuliaCon)

Not Present:
Manuel Rivas (Asst. Prof. Biomedical Data Science @ Stanford)
Julia Meinwald (Two Sigma—new member as of April)
Kari Jordan (Deputy Dir. Assessment for Data Carpentry)
Jennifer Klay (Prof. of Physics @ Cal Poly SLO, NumFOCUS Board of Directors)

Agenda

  1. Brief Re-Introductions
  2. Tasks a) Compile list of existing research and resources (we have a strong list already) -Gina, Erica b) Design PyData experiments (different tactics at different conferences; compare) -Reshama Postpone until data collection methodology is decided. c) Draft survey for PyData attendees (need to craft questions for assessment purposes)

Goal is to develop a model of what the barriers are and the channels causing those barriers. Collecting data, summary statistics, theoretical model, etc. Year 1 focus is data collection.

  • What demographics will we track?
  • What qualitative info do we want to know?
  • Pre- and post-surveys?

Erica: We could run a survey as part of JuliaCon (Jun 20-24, 2017). What do we want to learn from the survey? Designing survey questions is difficult, let’s start with the bigger picture.

Reshama: I can ask PyGotham organizers if they would like to participate. (should be Summer 2017).

Demographics: we want to know who is in attendance.

  • Did attendees feel comfortable, welcome?
  • Would attendees want to return, would they recommend it to others?
  • If you wouldn’t want to return or recommend, why not?
  • What would make the conference more welcoming, inviting?
  • Checklist? Which of the following things would be important for you to see at an event/want to take part in. Diversity events, childcare, ESL translation, specific policies, etc.
  • Why did you decide to come to this conference?
  • Were your goals/expectations met?
  • What other conferences / meetups / organizations do you participate in?
  • Did you submit to the CFP? Why/why not?
  • Is this your first event/technology event/professional/open source conference?
  • Did you receive financial support to come to this event? Would you have been able to come if you didn’t?

Erica: Survey responses are selective by people who attend and people who don’t. Of those who come, only some will respond.

Alex: For lazy survey-takers, keep it brief enough not to discourage people from answering.

Erica: Not more than 7 min.

Alex: 5-10 min.

Tobi: Questions that can be checkboxes instead of text.

Gina: Gender questions use an open text box ideally.

Tobi: How to incentivize responses? When people register, we get some data. Esp. if when I registered I also applied for financial assistance. For experimental design, consider some things. Make sure we’re not only getting responses from people who had a great time.

Erica: There’s data that we want to collect, but we deliberately made people reveal minimal info about themselves. This year they deliberately did not have people give their gender to make it more welcoming (don’t have to out yourself, etc.). Most important thing is to get the right people to the conference.

Tobi: Yes, there’s the idea of gathering minimal information to not make people uncomfortable, but there are always ways to balance need for info w/need for comfort of attendees. Don’t have to respond, make it optional. For attendees, more important to feel comfortable when you get there than for registration process to be comfortable. Why word of mouth, etc. is so important. Having a question about gender won’t turn off someone if the product (conference experience) comes highly recommended. What’s the main goal of the different events that occur at a conference and which take precedent. What attracts & retains a diverse population?

Leah: I agree.

Reshama: we cannot understand we cannot measure. If we don’t know our current diversity statistics, we won’t know if what we’re doing is working.

Erica: 100% agree with can’t understand what can’t measure. We’ll never be able to measure particularly well demographic composition of entire online community. To some extent, conference question e.g. how many women, hard to analyze if you don’t know if it’s proportional to the population.

Alex: Segmenting by gender responses, can pull a number of important things. Not just gender, financial support, whether goals & expectations were met, etc. Re: gender, may not have a global benchmark, but do have the possibility of longitudinal data. Can ask over any number of PyData conferences.

Tobi: What should registration process look like? What information should we glean from registrants? To Erica’s point, be mindful of users’ comfort levels, not scaring anyone off with weird questions. Things that are pretty standard; we know what that looks like. We’re also trying to be intentional and make sure registrants actually come, and then actually fill out the survey. Start to finish intentionally designing so the data we get is useful to answer the Q’s we have in an unbiased way.

Leah: Moore Foundation Kit is going to be used across all open source/data science events. These are suggested questions for your registration that can provide valuable input engaging your success on diversity & inclusion. In order to use JuliaCon, that’s in June. We would need to put together the survey fairly quickly to include that.

Erica: Would be great to send a survey immediately following the conference. Other thing to note, logistically, people who collect registration might not be the same as those who complete it.

Gina will work w/Erica to build a post-survey for JuliaCon and map out questions to be included within registration. Loop in Alex and the three of us discuss.

Week of May 22nd for next meeting.

Census considers new approach to asking about race – by not using the term at all
http://www.pewresearch.org/fact-tank/2015/06/18/census-considers-new-approach-to-asking-about-race-by-not-using-the-term-at-all/


NumFOCUS DISC Committee First Meeting - 03.28.17

(Diversity & Inclusion in Scientific Computing)  

Attendees:

  • Gina Helfrich (NumFOCUS Communications Director; DISC Program Mgr.)
  • Kari Jordan (Deputy Dir. Assessment for Data Carpentry)
  • Erica Moszkowski (Research Analyst, Fed. Bank of NY; graduate school in fall @ Harvard Business School, Diversity Chair @JuliaCon)
  • Reshama Shaikh (Data Scientist @ Metis; organizes NY WiMLDS)
  • Manuel Rivas (Asst. Prof. Biomedical Data Science @ Stanford)
  • Jennifer Klay (Prof. of Physics @ Cal Poly SLO, NumFOCUS Board of Directors)

Not Present:

  • Tobi Bosede
  • Alex Companioni

Agenda

  1. Brief Introductions
  2. Goals for the Moore Foundation 2-year DISC grant:
    • production of a scalable kit that can be disseminated to events to help promote best diversity and inclusion practices (including distribution plan)
    • assessment of current diversity programs at NumFOCUS
    • identify best diversity practices and challenges, plus suggestions for new programs
    • creation of initiatives including programs to increase the diversity of project contributors
  3. Overview of past NF diversity initiatives
    • Scholarships to PyData, other conferences (SciPy, LSST, AstroPy workshop, etc.)
    • CFP outreach (primarily to local organizations promoting UR’d groups, @callbackwomen etc)
    • Local conference committees tasked with creating diverse line-up for keynotes, invited talks
    • Local conference committees tasked with making sure their membership and review committee is diverse
    • Local workshops for high school girls on how to code/intro to data science
  4. Discussion: What do we mean when we say “diversity?”

Jennifer: OSS & Coding community is pretty homogenous, so any expansion of people who haven’t been traditionally well-represented.
Kari: What do we mean by events? (Gina: Moore Found. wants to distribute to all their communities for diverse & inclusive events)
Erica: Different OS communities have different norms & people from different backgrounds. E.g. career-changers, women of color, etc.
Jennifer: Bringing people from broader academic prep backgrounds into coding & data science.
Manuel: Community has an opportunity to make it obvious to others that you can do computing almost anywhere in the world... Resources you have to start with shouldn’t be a reason to keep you from the community. E.g. lower socio-economic status communities
Erica: Yes, that’s part of the reason we do Open Source (reduces costs, barriers to access)
Manuel: Data availability across many domains. E.g. rich biomedical or social network data
Reshama: How do we measure diversity? Do we have statistics for NF conferences? Increasing diversity at conferences or the general scientific community or going to the pipeline?
Gina: Diversity on the basis of typical privilege/oppression dynamics (socio-economic, dis/ability, race, gender, sexual identity, etc.) Also challenge of even collecting data in a responsible way.
Erica (?): Make sure we’re making connections/relationships with conferences and events to support our testing efforts.
Erica: What does research mean? Does it mean we have a model of what the barriers are and the channels causing those barriers? Can imagine collecting data, summary statistics, theoretical model, etc.
Gina: We could conduct research at different degrees of rigor.
Erica: Research for Year 1 focus should be data collection. Can we generate summary statistics and a menu of different barriers based on what we’ve observed. Could we run our own survey (e.g. at PyData)? Ask about not just who they are but also about participation in OS Community, more free-form responses, inquire about barriers faced, etc. Esp. w/ability to compare similarly-run conferences in different places.
Gina: Yes, we can put PyData resources and data-collection possibilities behind this effort.
Reshama: re first year = data collection; have almost a prospectus study where one PyData city (e.g. NYC) could have certain methods that are less conventional. Example if the goal is 50% women, reach out to various communities of women and asking them to participate and submit proposals. 2nd: Whole process of submitting to conferences is very well outline on e.g. O’Reilly page, but maybe for underrepresented folks the process needs to be different. Maybe the process needs to be different to attract more diverse participants. 3rd: Could do it for one city and not another to measure impact: invite a woman you’d like to see speak, participate, attend (to male participants) and see what results.
Erica: Does anyone have exp. running experiments?
Reshama: Has run clinical trials.
Gina: Has done qualitative research, could lend that to analyzing PyData survey.
Reshama: Can think of two people we could interview. CODE: Debugging the Gender Gap — was shown at Meetup headquarters; some of the engineers in the film work there. Have been able to hire a lot more female engineers. Read blog on founder of Meetup’s efforts to diversify company. Also knows data scientist at H20 in SF; if you really want to hire more women, it has to be a priority and can work.
Gina: Has resources/list to contribute to the survey of available research; Igor (not present) hopefully can also help re: barriers to OS participation.
Reshama comments to chat window:
Here's the blog:
https://backchannel.com/how-meetup-ditched-its-boys-club-4a3a3084e72f
We can also identify Men Who are "Advocates of Women"
Reshama: Once we define “diversity” for the group, perhaps start with one group and try to increase for that group, or aim at all groups? Define some strategy for that.
Erica: What do you mean by group?
Reshama: Gender, race, sexual identity, whether we want to try for all at once or start with one and expand it to other groups, etc.

  • How to collaborate?
  • Shared google docs (hosted @ NF)
  • Slack for conversation and ideas (NF can host it)
  • What are next steps?
  • [x] Gina will add members to Slack
  • [x] Gina will create GitHub file with roadmap
  • [x] Gina will create Google Docs for PyData survey + experiments
  • [x] Gina will create Google Doc for list of existing research, resources
  • [x] Establish near-term task list, ask everyone to self-sign up on what they want to contribute to

Proposed Roadmap, Year 1:

  • April-June
    1. Compile and collate relevant research and existing data on diversity in open source and scientific computing communities (Include assessment of current/past NF initiatives)
    2. Summarize barriers to collection of accurate data (Gina will give a talk on this subject at PyData Seattle)
  • July-September
  1. Analyze research + data to identify barriers to full participation, gaps in research
  2. Formulate strategy to tackle barriers to collection of accurate data
  • October-December
  1. Conduct research to fill any major gaps in understanding participation, barriers to participation
  2. Develop initial list of best practices (proven and/or untested)
  3. Devise rollout method for testing best practices

Ensure we are communicating with conferences available for that.

  • January-March
  1. Conduct best practices test(s)
  2. Collect & analyze results
  3. Revise best practices list as-needed
  4. Publish summary of research and findings