Call the correct API to determine if a user is in treatment or control group #20183

Activity

1. Luciana Abud (luabud)

   added

   debtCode quality issuesCode quality issues

   area-internalLabel for non-user facing issuesLabel for non-user facing issues

   needs spikeLabel for issues that need investigation before they can be worked on.Label for issues that need investigation before they can be worked on.

   on Nov 8, 2022
2. Kartik Raj (karrtikr)

   added

   importantIssue identified as high-priorityIssue identified as high-priority

   on Nov 8, 2022
3. luabud commented on Jan 27, 2023

   

   Luciana Abud (luabud)

   on Jan 27, 2023

   MemberAuthor

   More actions

   It really should be getExperimentValue: https://github.com/microsoft/tas-client/tree/main/vscode-tas-client#use
4. Luciana Abud (luabud)

   closed this as completedon Jan 27, 2023
5. karrtikr commented on Jan 31, 2023

   

   Kartik Raj (karrtikr)

   on Jan 31, 2023

   More actions

   Great, we don't have an issue tracking implementing this though, so can we reopen this?

   👍React with 👍1Reacted by Luciana Abud
6. Luciana Abud (luabud)

   removed

   needs spikeLabel for issues that need investigation before they can be worked on.Label for issues that need investigation before they can be worked on.

   on Feb 3, 2023
7. Luciana Abud (luabud)

   reopened this on Feb 3, 2023
8. github-actions

   added

   triage-neededNeeds assignment to the proper sub-teamNeeds assignment to the proper sub-team

   on Feb 3, 2023
9. Luciana Abud (luabud)

   changed the title [-]Spike - what is the right API to call for retrieving experiments [/-] [+]Use getTreatmentVariable instead of isInExperiment for running experiments [/+] on Feb 3, 2023
10. luabud commented on Feb 3, 2023

    

    Luciana Abud (luabud)

    on Feb 3, 2023

    MemberAuthor

    More actions

    Oops, good point :) thank you!
11. Karthik Nadig (karthiknadig)

    assigned

    Kartik Raj (karrtikr)

    on Feb 6, 2023
12. karrtikr commented on Feb 8, 2023

    

    Kartik Raj (karrtikr)

    on Feb 8, 2023

    More actions

    Luciana Abud (@luabud) Thing is inExperiment already uses getTreatmentVariable 🤷‍♂️:
    

    vscode-python/src/client/common/experiments/service.ts

    Line 143 in e92c124

    const treatmentVariable = this.experimentationService.getTreatmentVariable(EXP_CONFIG_ID, experiment);

    See More

    So back to square one, not sure what is to be used here.
13. Kartik Raj (karrtikr)

    changed the title [-]Use getTreatmentVariable instead of isInExperiment for running experiments [/-] [+]Spike - what is the right API to call for retrieving experiments[/+] on Feb 8, 2023
14. Kartik Raj (karrtikr)

    added

    needs spikeLabel for issues that need investigation before they can be worked on.Label for issues that need investigation before they can be worked on.

    and removed

    triage-neededNeeds assignment to the proper sub-teamNeeds assignment to the proper sub-team

    on Feb 8, 2023
15. Kartik Raj (karrtikr)

    removed their assignment

    on Feb 8, 2023
16. luabud commented on Feb 10, 2023

    

    Luciana Abud (luabud)

    on Feb 10, 2023

    MemberAuthor

    More actions

    Kartik Raj (@karrtikr) but it returns treatmentVariable !== undefined;, which is true whether user is in treatment or control, no?
17. Kartik Raj (karrtikr)

    self-assigned this

    on Feb 13, 2023
18. Kartik Raj (karrtikr)

    added this to the February 2023 milestone on Feb 13, 2023
19. karrtikr commented on Feb 13, 2023

    

    Kartik Raj (karrtikr)

    on Feb 13, 2023

    Last edited by karrtikr

    More actions

    I think you were the one who observed this right? So as I understand the problem here is that how do we distinguish between control groups and treatment groups.

    Digging a little bit:

    call getTreatmentVariable(configId: string, name: string) to get the value of a treatment variable. configId is always vscode and name is the name of the treatment variable that you defined in control tower when you set up your experiment.

    The doc says getTreatmentVariable returns the value of the treatment variable. And when we create an exp on control tower:

    

    We generally set the value of treatment as true for experiment group, and false for control group. Given we always follow this convention, I think we should be checking if treatmentVariable === true instead. Does it make sense?

    👍React with 👍1Reacted by Luciana Abud
20. Kartik Raj (karrtikr)

    closed this as completedin #20690on Feb 13, 2023
21. Kartik Raj (karrtikr)

    added a commit that references this issue on Feb 13, 2023

    Call the correct API to determine if a user is in treatment or contro…

    ...

    2202fbe
22. Kartik Raj (karrtikr)

    changed the title [-]Spike - what is the right API to call for retrieving experiments[/-] [+]Call the correct API to determine if a user is in treatment or control group[/+] on Feb 13, 2023
23. Karthik Nadig (karthiknadig)

    added

    verifiedVerification succeededVerification succeeded

    on Feb 21, 2023
24. github-actions

    locked as resolved and limited conversation to collaborators on Mar 24, 2023
25. Wes McKinney (wesm)

    added 2 commits that reference this issue on Mar 28, 2024

    Merge latest upstream python extension

    ...

    34c70bb

    Merge latest upstream python extension

    ...

    8292d65