We are having problems reading the contents of the “Readme.txt” file. What does it say?

Here is all of the information in that file:

• 2023_Competition_Training.csv       = Data to be used for analysis & model development
• 2023_Competition_Holdout.csv       = Holdout data to be scored with final model and results returned for mid-cycle leaderboard and/or Oct 16 submission
• Humana_Mays_2023_DataDictionary.xls       = File Statistics, File Layout, descriptions of attributes for each event type

When will the dataset be available?

• Competition data will be distributed to registered and verified teams starting after September 13, 2023, Informational Call and ending after the registration deadline of September 22, 2023.  (Typically, data will be available no more than 48 hours after registration & verification)

What format will the dataset be in?

• The dataset will be available in a CSV file, along with a data dictionary.
• Are we allowed to use publicly available data to help us in this case competition?
• Yes. Students are encouraged to use open-source data when creating a solution.

Explain why there are instances in the claims data where the process date is less than the service date. 

• You are correct in that the process date should not occur prior to the service/visit date.  2 things may be happening: (a) there is a problem with your data (i.e. read-in or join error) (b) the data is erroneous.  Data is messy. You must decide which it is and how to handle it.

What is the distinction between “medclm_key” and “clm_unique_key” in the context of the “medclms_trian” dataset, and why does “medclm_key” seem to have a unique value for every row while “clm_unique_key” has duplicate values?

• You should think of medclm_key as the primary key for the medical claims table and it is unique for every claim line.  The clm_unique_key groups together a single “claim” which can consist of multiple “claim lines”.  These unique claims can be combined together to form a logical claim that group together claims from the same provider/member combo with overlapping service dates.  We typically use the logical claim to count utilization/visits rather than clm_unique_key.

Are we allowed to utilize a Private Github repository to share access to data between team members?

• Yes. However, make sure that data is not public as it would result in a violation of NDA.

Are the two claim datasets claims filed by healthcare providers or by patients?

• The way this usually works for medical claims is this: Someone goes to the doctor, they show their insurance card to the doctor, and after the appointment, a billing person submits a claim to the insurance company.
• For prescription claims, it’s a lot faster but generally the same process. When a patient fills a claim at a pharmacy, the pharmacy submits the claim to the insurance company.
• Since Humana is the insurance company, this is the data we have. Generally, it comes from the provider or pharmacy directly to Humana.

How much on average does the insurance cover for patients in both medical claims and prescription drugs? Do participants have access to that data? 

• Coverage amounts vary on myriad of factors including, but not limited to, the plan an individual member has chosen.  However, for the purpose of this case competition, those details are not included in the data that was provided to the participants.

 

Why might one prioritize AUC over recall in this context? To elaborate, when we focus on maximizing recall, there is a slight decrease in AUC, but we end up capturing a larger portion of the positive class. Despite the model’s lower precision, the subsequent false positives we investigate could still be advantageous, even if we acknowledge that they might not influence the treatment decision

• It’s not that we believe AUC is more important, it is simply the measure of accuracy chosen for Round 1 evaluation where every team’s model will be evaluated using the same measuring stick.  However, feel free, in subsequent rounds, to make a case for a model that maximizes recall, why that makes sense, and what (if any) implications it carries.
• What are the ways humana is currently intervening with patients that seem to abandon the treatment prematurely? And how effective are they?
• As a specialty pharmacy, CenterWell Specialty Pharmacy (CWSP) offers a patinet management program to improve adherence to specialty therapies. This includes side effect management and adherence strategies. We have an accredited Oncology Center or Excellence (or Cancer Center of Excellence) to support patients filling oral oncology medications with CWSP. For Round 2 proposals, consider innovative ways both CWSP (as a specialty pharmacy) and Humana (as a plan) can solve for the business problem.

How do you know if Tagrisso is working or not? Based on what metric?

• Typically oncologists will monitor based on imaging to judge the response. They assess if the tumor(s) are shrinking, are stable, or are growing. In clinical trials, this is called objective response rate (ORR) and typically a secondary outcome, after progression-free survival (PFS) or overall survival (OS). World Health Organization and Response Evaluation Criteria in Solid Tumors (RECIST) are anatomic response criteria to determined ORR.
• Is the Tagrisso treatment at the same dosage for all the patients.
• The starting dose is 80 mg by mouth once daily. The Tagrisso PI contains additional details related to dose adjustments for adverse effects and/or drug-drug interactions.

Is Tagrisso a generic form of Osimertnib?

• Tagrisso is the brand name. The generic name is osimertinib. It’s a part of the drug class epidermal growth factor receptor (EGFR) tyrosine kinase inhibitor (TKI).

What are the costs associated with extra follow-up (money, time, effort)? Are there constraints? 

• It will highly vary based on the patient, the patient’s insurance plan/benefits, physical location, social determinants of health, etc. This would be something to possibly include in Round 2 proposals.

What are maintenance vs. non-maintenance drugs?

• Maintenance medications are used for chronic or ongoing conditions, typically ones requiring treatment for months or years or even indefinitely. Examples include: high blood pressure, diabetes, and high cholesterol. Non-maintenance or acute medications are typically for short-term use. Examples include: antibiotics, anti-nausea medications, post-surgery medications. Osimertinib would be considered a maintenance medication; we expect (hope) the patinet stays on therapy until disease progression or unexpectable toxicity (for metastatic lung cancer) or for 3 years (for resected lung cancer)

Is there more information about Tagrisso and how it is administered that you can provide?

• More information on Tagrissio can be found in the following fact sheet : Tagrisso PI 6.2023
• Based on the data dictionary, the ‘util_cat’ variable is linked with ‘admit_type’ and ‘pot’. Can you explain what ‘admit_type’ represents?
• ‘admit_type’ is another variable similar to ‘util_cat’ and ‘pot’ which we removed before publishing the competition data. You shouldn’t need to worry about it.

What is the relationship between service date and process date and which date is closer to the actual drug administer date?

• The service date corresponds to the date a pharmacy attempted to adjudicate a claim. This is not the same thing as the date an individual may have picked up a prescription or started taking a therapy (neither of which we could know), though in many cases in that may be what happens and theory tells us they should be highly correlated. The process date is simply when Humana became aware of this attempt. Generally it is the day after the service date, but in some cases (usually the result of certain administrative actions) we may receive updated data about an attempt later. On should consider the process date being the earliest date where that data would be available to use in a predictive model
• In the train dataset a few people are labelled as ceasing the treatment early, yet they do not have any ade diaognosis listed. The project prompt states that unsuccessful therapies are only those that end prematurely and have an ade reported. Is there something wrong there? If the difference between therapy end and start is 2 months, but the target indicator is 0, does that mean he discontinued but there was no ADE so we don’t need to predict it?
• The target you want to predict is those therapies that end early and have an ADE diagnosis. All other situations, including those in this question, are not part of the target population.
• We don’t have any way to show a causal relationship between the ADE and the end of therapy, which is why we define the target as ‘experience an ADE and discontinue,’ not ‘experience an ADE which causes discontinuation.’

Why therapy start and end date doesn’t match with visitor dates in med claim? Our assumption is a pharmacy claim follows a visit, and the service date is equal to the visit date. Can anyone verify this assumption? every visit might result in a pharmacy claim, but can a pharmacy claim happen without any visit ? and should we allow the service date to be in a few days range from the visit date or should be the same day ?

• There is no guaranteed or formal or required relationship between a medical claim and a pharmacy one.
• The diagnosis indicators at the end of the medclms file, are these already mapped from the diagnosis codes (ICD-10) in the same file? and what year ICD-10 codes are being used? Is the nausea_diagnosis marked against each patient or each claim?
• The diagnosis code indicators included in the med claims dataset use the diagnosis codes from the same row. When one of those codes matches a code for one of the indicated diagnoses, that indicator gets a ‘1’ value.
• If there is a ‘1’ value for any of the diagnosis indicators, you also have a ‘1’ for the ADE indicator. An ADE indicator of 1 means that one of the diagnosis codes reported on that claim is on a list of diagnosis codes related to known side effects (ADEs) of Osimertinib.

What is metric_strength? Is there a “universal” measure for drugs that we could use to homogenize?

• Use both the ‘metric_strength’ and the ‘strength_meas’ data fields.
• Could you please explain about therapy_end_date not being available in holdout how to learn from target_train and not find it in holdout file?
• In the train datasets, you have all the information we have for those members before and during their therapy. In the holdout datasets, you have all the data from before their therapy up to a randomly generated cutoff date during their therapy. This cutoff date isn’t specified for you.
• Providing the end date of the therapy would be a type of ‘target leakage’. If you knew when a therapy was going to end, you would know if it ended or not. Also, for the models to be useful, they need to predict our target during the therapy before we have the end date of the therapy.

As mentioned in the TAMU predictive modeling prompt: “All pharmacy and medical claims contain simplified information for an individual during the time 90 days before their Osimertinib therapy and through the end of therapy.” In this case, can we consider the ‘therapy_start_date’ of each patient as the moment at which the model predicts the Target?

• It’s up to you to determine what would be an appropriate prediction date. In the train datasets, you have all the information we have for those members before and during their therapy. In the holdout datasets, you have all the data from before their therapy up to a randomly generated cutoff date during their therapy. This cutoff date isn’t specified for you.
• Is the holdout set stratified on the target variable?
• The target variable is not included in the holdout set in any way. If you’re asking if the proportions of positive and negative cases are the same between the training data and the hold out data, that information is not provided for this competition

Can you provide more information on what pay_day_supply_cnt is?

• Pay_day_supply_cnt is the number of days of medication covered by a prescription fill. Often, this is 1 pill/treatment per day for 30 days. Pay_day_supply_cnt=30. i.e. you have enough medication for 30 days.

‘maint_ind’ – What is a maintenance drug?          

• This is a heuristic indicator that the specific drug is often used for chronic conditions requiring ‘maintenance’ or continued treatment.

 ‘specialty_ind’- What is a specialty drugs?

• The specialty_ind is an internal-to-Humana approach to labeling if a given drug would be considered a specialty therapy. There is no universal definition of what would be considered a specialty drug, and usually these determinations are made in part for operational or contractual considerations.
• There is no universal definition of a specialty drug. A directional concept would be that specialty drugs are typically much more expensive than non-specialty medications. The specialty drug indicator provided is an internal-to-Humana definition that spans several concepts/criteria and is not appropriate to use in all cases

Are we allowed to use any external datasets apart from the ones that are provided ?

• Yes.

We find some ndc_ids starting with ‘11111’ with missing values, but we can’t find any info from the National Drug Code Directory. Do you have any idea about it?

• NDC’s beginning with 11111 correspond to an OTC medication covered by the members benefit. We do not provide a crosswalk to which OTCs the member may have received (nor any guarantees of a 1:1 relationship between the NDC and a specific OTC product)

What is the relationship between pay_day_supply_cnt and rx_cost in rxclms?  With respect to tot_drug_cost_accum_amt, how can we connect this with pay_day_supply_cnt and rx_cost?

• There is no direct formal relationship between pay day supply count and the cost of the medication

Are all the claims approved?

• You can assume that all claims in this dataset have been approved.

Could you please provide more information about the ‘diag_cd#’ column in the ‘medclms’ file? I noticed that there are multiple columns with names like ‘diag_cd1,’ ‘diag_cd2,’ and so on. What do these columns represent, and how are they used in the context of our project? In the medical claim dataset, there’s both primary and secondary diagnosis. The question is: Is there a difference between the two categories as far as their relation with the visit?

• Any medical claim can have multiple diagnoses associated with it, and each diag_cd# column is space for one code to be listed.
• If a user_id / therapy_id does NOT have an entry in the med (or) Rx claims – can I say that this person did not have any medical visit / prescription? Or should I assume that this could be missing data? In both train and holdout datasets, there are patients with neither medical claim records or pharmacy claims. For those cases, is it acceptable to remove such patients from both training and scoring? Or, would that impact scoring calculations?
• Every individual in the target dataset should have corresponding rx and/or med claims. If you find examples of when that’s not the case, use your best judgement to decide what to do.
• Do we have to predict a likelihood percentage or a classification variable (0 or 1)?
• Predictions should be a probability between 0 and 1 for the Round 1 Submission so we can calculate an AUC. Teams may recommend a classification strategy or other performance metrics as part of their submissions in Round 2.

Overall I am confused with why we are predicting a probability for unsuccessful therapies rather than an outright classification as 0 or 1. Why is that so? If we have model A that scores everyone who ceased treatment as 0.9 and others as 0.1, and we have model B that scores them all as 0.8 and 0.2 respectively, would these models receive the same Accuracy score since they classify everyone correctly?

• Please review ROC and AUC metric calculations

What to do with protected attributes, how does it relate with equity?

• Please see Humana’s policy on Responsible AI:  see attached document

Can we get the humana’s Osimertininb linked side effects,Rx Claims, Med Claims costing per annum? overall costing for humana for this lung cancer health care model?

• This data will not be provided for this competition. Teams may supplement what is provided with publicly available data sources