Week 2: Cracking LineUp

Last week, I took a bit of a break from my previous task. Instead, I joined forces with the rest of my team to figure out the LineUp codebase. We met on Friday and Saturday in a hackathon-style event and worked together to read through all the files.

What we learned

LineUp was written in TypeScript, which is a version of JavaScript with type-checking. There are 6 typescript files in the src folder which are briefly described below. The codebase we downloaded came with a node_modules folder that has 655 different packages.

app.ts

This is the largest typescript file in this folder. Includes methods for fixing missing data values and initializing LineUp with a header bar and dataset (calls array of datasets from datasets.ts).

datasets.ts 

Specifies an array of datasets that are included in the demo folder. This array is loaded by app.ts.

export.ts

Includes methods for exporting to CSV and exporting to JSON.

gist.ts

Includes methods for loading and saving to GitHub Gists.

importer.ts

Includes methods for importing a file, parsing the JSON, and converting to columns. Called from app.ts.

index.ts

Import statements for all the icons and pretty html formatting.

node_modules

From what I can tell, only 4 of these are being used. If possible, we will try to remove some of these packages to make the project smaller.

Testing with my own data

I tried creating a dummy dataset as a csv file and uploading it to LineUp. The following is a snapshot of my dataset in the app.

However, when I export this dataset as a JSON file and the re-upload it, I get the following:
This issue is semi-reproducible, meaning that sometimes it works and sometimes it doesn't.

Next steps with LineUp

There are a couple issues we need to address before we can use this application on our website:
  • I think some of the typescript files should be reorganized. For example, missing values should not be fixed in app.ts where the project is loaded. It makes more sense for that to be in import.ts.
  • We need to remove the bug listed in "Testing with my own data"
  • We need to start LineUp with our own JSON file from a central data repository.
  • We need to remove the download and upload functionality and instead have a "save" button that returns a JSON file to a central data repository.

Plan of attack

In the next week, I am acting as architect and task manager. I created the following business logic diagram to start the conversation with my teammates about what we need to work on.
I tasked myself with understanding the LineUp codebase and determining what changes will need to be made to make it functional for our project. 

Two of my teammates have asked to be more involved in the research side as well. Frankly, I think it is a great idea to have multiple heads working on the problem of building and explaining a rank. I look forward to working with them on this. I created a library of references on Zotero for them to peruse, and we will start talking about the research tomorrow.

Comments

Post a Comment

Popular posts from this blog

Week 31: Datasets for fairness

This week, Diana and I looked for some datasets on which to evaluate our fairness criteria. This is no simple task. For one, the datasets need to be large enough to be split into training, calibration, and testing sets. Second, there needs to be some protected group attribute like sex, race, or age. Third, there needs to be some outcome ranking as well as some truth value. The truth is often the hardest attribute to find. In this post, I introduce some of the datsets we decided to use. I try to identify each of the necessary features that we need to properly test our fairness criteria. COMPAS The COMPAS dataset is an obvious choice because it is well ingrained in the literature about fairness. It has 18,000 observations (n=18000) and 1000 attributes (k=1000). Protected groups can be sex (81/19) or race (51/49). The outcome attribute can be the recidivism score, the score used in court that represents the risk that the accused will commit another crime. The truth attribute is a li...
Read more

Week 32-33: Wrapping up

Hi all. This is my last blog post for this project. The last few weeks, we spent little time working on the project and instead focused on finishing up our classes. To be honest, we lost some motivation after finishing the paper, and we've been reluctant to start up new tasks so close to the end of the year. In this post, I'd like to talk about some of the things I've learned in this project other than what we found in our research. This post is really more a service to myself; to help me reflect on this past year and what I've learned. When I started this project, I didn't understand how research in computing worked. My exposure to research had been more biology/clinical, in which research involved setting a hypothesis about some protein, designing an experiment with positive and negative controls, and then running the experiment repeatedly until confident in the results. In computing research, my experience has been of two varieties: reading papers and proce...
Read more

Week 30: Returning to Fairness

With the LineUp paper submitted, it's now time to return to the fairness problem we were working on before. As a bit of a refresher for you and for me, I'll review what we've done with fairness so far and what remains to be done. Fairness in Ranking Up until now, the literature regarding fairness has focused on fairness in classification. As an illustrative example, consider fairness in hiring practices with repect to applicant sex. A job opening might receive 100 applicants evenly split into 50 males and 50 females. If males and females are equally capable of doing the job, then you might be concerned to see the hiring company mostly interviewing male applicants. To evaluate fairness, you would compare each applicant's capability (capable of performing duties, not capable) to their outcome (interviewed, not interviewed). In a completely fair scenario, the same proportion of capable applicants from each sex group would receive an interview. Small deviances from this...
Read more