Skip to content

Add notice about https://github.com/stackabletech/hdfs-operator/issues/274 containing a breaking change to the release notes #348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fhennig merged 1 commit into from
Jan 26, 2023
Merged

Add notice about https://github.com/stackabletech/hdfs-operator/issues/274 containing a breaking change to the release notes #348

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions modules/ROOT/pages/release_notes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,54 @@ to
zookeeperConfigMapName: simple-druid-znode
```

==== Stackable Operator for Apache Hadoop
* https://github.com/stackabletech/hdfs-operator/issues/274[Support for multiple storage directories]

As part of the change mentioned above the naming scheme for the PersistentVolumeClaims written for DataNodes has been changed, so that PVCs written by earlier operator versions are not recognized any more.
Previous PVCs were called `hdfs-datanode-default-0` but now need to have the prefix `data-` added, because otherwise there may be naming collisions due to the ability to specify multiple storage classes.

In order to move over existing PVCs a few migration steps are required.
Since it is not possible to rename PVCs you'll need to delete the existing PVCs and recreate them with the correct name and bound to the correct backing PV.

Please find an example workflow of how this can be achieved below, this also sets the reclaim policy for the backing PV to `Retain` to avoid the PV being deleted when it becomes unbound, depending on your Kubernetes config this step may not be necessary.

[source,bash]
----
export PVNAME=$(kubectl get pvc hdfs-datanode-default-0 -o yaml | yq '.spec.volumeName')
Copy link
Copy Markdown
Member

@maltesander maltesander Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would mention the yq version. For me on 4.17 i have to do (this solution requires 4.18+):
export PVNAME=$(kubectl get pvc hdfs-datanode-default-0 -o yaml | yq e '.spec.volumeName' -)
and maybe add the namespace like:
export PVNAME=$(kubectl get -n <namespace> pvc hdfs-datanode-default-0 -o yaml | yq '.spec.volumeName')

Copy link
Copy Markdown
Contributor

@fhennig fhennig Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, too late


kubectl patch pv ${PVNAME} -p '{"spec":{"persistentVolumeReclaimPolicy": "Retain"}}'

kubectl delete pvc hdfs-datanode-default-0

kubectl patch pv ${PVNAME} -p '{"spec":{"claimRef": null}}'
----

Afterwards you can recreate the PVC with the new name and bind it to the PV.

Please note that you will need to adapt labels, storageClassName and resources to your specific configuration.
Ideally export the pre-existing PVC with kubectl and change the name.

[source,yaml]
----
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app.kubernetes.io/component: datanode
app.kubernetes.io/instance: hdfs
app.kubernetes.io/name: hdfs
app.kubernetes.io/role-group: default
name: data-hdfs-datanode-default-0
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: standard
volumeMode: Filesystem
volumeName: <insert PV Name here>
----

==== Stackable Operator for Apache Hive
* https://github.com/stackabletech/hive-operator/pull/292[Moved database specification from role/role-group level to top-level clusterConfig]
Expand Down