Skip to content

Fix validator bug where list_of_list conversion didn't capture inner list type #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hdunham merged 4 commits into from
Mar 25, 2022
Merged

Fix validator bug where list_of_list conversion didn't capture inner list type #308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ffec717
fix validator bug where list_of_list conversion didn't capture inner …
hdunham Mar 8, 2022
f289c5c
fix CP timestep database field type
hdunham Mar 8, 2022
9859f0b
remove repeat attempt to convert to list_of_list
hdunham Mar 8, 2022
2690337
Update CHANGELOG.md
hdunham Mar 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,20 @@ Classify the change according to the following categories:
##### Removed
### Patches

## Develop - 2022-03-08
### Patches
- `reo`: Fix list_of_list conversion in `validators.py` not capturing inner list type. E.g. a 2D list of floats that was supposed to be a 2D list of integers wasn't getting caught.

## v1.9.1 - 2021-12-16
### Minor Updates
##### Added
- `reo`: `GHP` heating and cooling HVAC efficiency thermal factor inputs and defaults to account for a reduction in heating and/or cooling loads with GHP retrofits
- `*.jl`: Reduction in heating and cooling loads due to the thermal factors (described above) if `GHP` is chosen.
- `reo`: **GHP** heating and cooling HVAC efficiency thermal factor inputs and defaults to account for a reduction in heating and/or cooling loads with **GHP** retrofits
- `*.jl`: Reduction in heating and cooling loads due to the thermal factors (described above) if **GHP** is chosen.

## v1.9.0 - 2021-12-15
### Minor Updates
##### Added
- `reo`: Added capability to estimate year 1 and lifecycle emissions and climate and health costs of CO2, NOx, SO2, and PM2.5 from on-site fuel burn and grid-purchased electricity. Added total renewable energy calculations. User options to include climate and/or health costs in the objective function. User options to set emissions and/or renewable electricity constraints. User options to include or exclude exported electricity in renewable energy and emissions calculations. New emissions and renewable energy inputs (and defaults) in `nested_inputs.py` and outputs in `nested_outputs.py`. Added default emissions data for NOx, PM2.5, and SO2 in `src/data`. Default marginal health costs in `src/data/EASIUR_Data`. In `views.py` and `urls.py` added `easiur_costs` and `fuel_emissions_rates` urls. Default fuel emissions rates for NOx, SO2, and PM2.5 in `validators.py`. Added calculation of breakeven CO2 cost (when NPV is negative).
- `reo`: Added capability to estimate year 1 and lifecycle emissions and climate and health costs of CO2, NOx, SO2, and PM2.5 from on-site fuel burn and grid-purchased electricity. Added total renewable energy calculations. User options to include climate and/or health costs in the objective function. User options to set emissions and/or renewable electricity constraints. User options to include or exclude exported electricity in renewable energy and emissions calculations. New emissions and renewable energy inputs (and defaults) in `nested_inputs.py` and outputs in `nested_outputs.py`. Added default emissions data for NOx, PM2.5, and SO2 in `src/data`. Default marginal health costs in `src/data/EASIUR_Data`. In `views.py` and `urls.py` added **easiur_costs** and **fuel_emissions_rates** urls. Default fuel emissions rates for NOx, SO2, and PM2.5 in `validators.py`. Added calculation of breakeven CO2 cost (when NPV is negative).
- `reopt_model.jl`: Included additional optional constraints for emissions reductions (as compared to BAU) and renewable electricity percentage. Added optional inclusion of climate and health costs to the model objective and associated life cycle cost calculation. Added calculations of life cycle emissions and costs for CO2, NOx, SO2, and PM2.5. Added calculation of renewable energy (% and kWh), which includes electric and thermal end uses. Emissions and renewable energy calculations account for all technologies.
- `utils.jl` added emissions- and renewable energy-specific parameters.
##### Changed
Expand Down
2 changes: 1 addition & 1 deletion reo/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,7 +410,7 @@ class ElectricTariffModel(models.Model):
chp_standby_rate_us_dollars_per_kw_per_month = models.FloatField(blank=True, null=True)
chp_does_not_reduce_demand_charges = models.BooleanField(null=True, blank=True)
emissions_region = models.TextField(null=True, blank=True)
coincident_peak_load_active_timesteps = ArrayField(ArrayField(models.FloatField(null=True, blank=True), null=True, default=list), null=True, default=list)
coincident_peak_load_active_timesteps = ArrayField(ArrayField(models.IntegerField(null=True, blank=True), null=True, default=list), null=True, default=list)
Copy link
Copy Markdown
Member

@NLaws NLaws Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this change require a migration?

coincident_peak_load_charge_us_dollars_per_kw = ArrayField(models.FloatField(null=True, blank=True), null=True, default=list)
emissions_factor_CO2_pct_decrease = models.FloatField(null=True, blank=True)
emissions_factor_NOx_pct_decrease = models.FloatField(null=True, blank=True)
Expand Down
4 changes: 2 additions & 2 deletions reo/nested_inputs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ def list_of_float(input):
def list_of_str(input):
return [str(i) for i in input]

def list_of_list(input):
return [list(i) for i in input]
def list_of_list(input, inner_list_conversion_function=list):
return [inner_list_conversion_function(i) for i in input]

def list_of_int(input):
result = []
Expand Down
2 changes: 1 addition & 1 deletion reo/tests/test_coincident_peak.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def setUp(self):
"doe_reference_name": "MidriseApartment"
},
"ElectricTariff": {
"coincident_peak_load_active_timesteps": [[1,2,100],[6000,7000]],
"coincident_peak_load_active_timesteps": [[1,2.0,100.0],[6000.0,7000.0]],
"coincident_peak_load_charge_us_dollars_per_kw": [10,5],
"add_blended_rates_to_urdb_rate": False,
"net_metering_limit_kw": 0,
Expand Down
38 changes: 21 additions & 17 deletions reo/validators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2018,12 +2018,15 @@ def convert_data_types(self, object_name_path, template_values=None, real_values
:return: None
"""

def test_conversion(conversion_function, conversion_function_name, name, value, object_name_path, number, input_isDict, record_errors=True):
def test_conversion(conversion_function, conversion_function_name, name, value, object_name_path, number, input_isDict, record_errors=True, list_of_list_inner_conversion_function=None):
try:
series = pd.Series(value)
if series.isnull().values.any():
raise NotImplementedError
new_value = conversion_function(value)
if list_of_list_inner_conversion_function == None:
new_value = conversion_function(value)
else:
new_value = conversion_function(value, inner_list_conversion_function = list_of_list_inner_conversion_function)
except ValueError:
if record_errors:
if input_isDict or input_isDict is None:
Expand Down Expand Up @@ -2094,24 +2097,25 @@ def test_conversion(conversion_function, conversion_function_name, name, value,
# Finally if it is a valid alternate type we set it to be converted to a list at the end
# otherwise we flag an error
try:
new_value = test_conversion(list_of_list, "list of list", name, value, object_name_path, number, input_isDict, record_errors=False)
new_value = test_conversion(list_of_list, "list of list", name, value, object_name_path, number, input_isDict, record_errors=False, list_of_list_inner_conversion_function=eval(list_eval_function_name))
except:
isValidAlternative = False
for alternate_data_type in attribute_type:
try:
new_value = eval(alternate_data_type)(value)
attribute_type = alternate_data_type
make_array = True
# In case where the data is not at least a list (i.e. int), make it a list
# so it will later be made into a list of lists
if not isinstance(new_value, list):
make_array_of_array = True
new_value = new_value
self.update_attribute_value(object_name_path, number, name, new_value)
isValidAlternative = True
break
except:
pass
if alternate_data_type != "list_of_list":
try:
new_value = eval(alternate_data_type)(value)
attribute_type = alternate_data_type
make_array = True
# In case where the data is not at least a list (i.e. int), make it a list
# so it will later be made into a list of lists
if not isinstance(new_value, list):
make_array_of_array = True
new_value = new_value
self.update_attribute_value(object_name_path, number, name, new_value)
isValidAlternative = True
break
except:
pass
if isValidAlternative == False:
if input_isDict or input_isDict is None:
self.input_data_errors.append('Could not convert %s (%s) in %s to one of %s' % (
Expand Down