As it’s coming up on my one year anniversary of being a Security Engineer, basically a mix of a software engineer and data engineer for our InfoSec teams, I figured it would be a good time to reflect on all that this year has been, including the good, the bad, and the ugly.

• The good, my role offers me the type of variety and challenge I generally crave.

• The bad, it’s often difficult to feel like you’re making any progress when it feels like you frequently have no idea what you’re doing and there’s always more to learn.

• The ugly, I’ve definitely cried more in the past year than I have in any other working period just from the feelings of frustration, self-doubt and moments of being overwhelmed by the amount of complexity working on enterprise applications.

If it were not for my support system, including the fellow engineers on my team and my mentors, I think I would have given up but they’ve provided:

• A supportive learning environment, with room for reflection and a no-blame culture
• An outlet for sharing my frustrations
• An open door when I needed to think through my life (career) choices
• The encouragement needed to get my feet back under me and regain my confidence when going through a challenging bit of work (and made me feel like it was okay even if I didn’t that day)
• Still have the patience to talk me through and explain why my tests are failing or falling over for the nth time when I get stuck.

Things I’ve Learnt

I found it helpful to keep a list of what I was working on and learning, as day to day it can feel like you’re not making progress - but trust me, you are (and I was!)

Skills

Programming

My proficiency in both writing and reviewing Python code has greatly improved, and I’m starting to take in and follow some of the common design patterns and best practices through my work on:

• Our Security Engineering products and applications
• Deploying third party broker services to support our wider Product Security teams and their secure code tooling
• Debugging and test writing, including the use of mocking
• Implementing relevant logging, monitoring, and alerts
• Code reviews
• Investigation and resolution of production incidents
• Coursera and LinkedIn Learning courses on Python programming, Design Patterns, and Object-Orientated Programming (OOP) Best Practices.

Data Engineering

• Data retrieval via 3rd party APIs using API clients and client wrappers
  • GraphQL
  • REST APIs
• Data extraction, cleaning and transformation using Python (including pandas)
• Defining schema for tables in GCP
• Programmatic querying of databases using AWS Redshift (Postgres)
• Creation of Looker dashboards (data visualisation)
• Experimentation with:
  • Developing Bethos data pipelines
  • Querying data with Athena
  • Using pydantic for data validation and settings management
  • AWS Gameday: Generative AI Unicorn Party using Amazon Bedrock (Knowledge Bases and Agents) and Lambda Functions

Technologies

AWS

- IAM policies and resource permissions
- Elastic Kubernetes Service (EKS)
- Redshift
- Event Bridge
- SNS/SQS
- Lambdas
- S3

GCP

- IAM policies and configuration for Service Accounts
- Cloud Functions
- Cloud Scheduler
- Pub/Sub
- Google Cloud Storage (GCS)
- BigQuery
- Secrets Manager
- Monitoring and Alerts

IaC

- Terraform
- Helm
- Kubernetes (YAML)

Other:

- VS Code
- GitHub
- Secrets Management
- CI/CD using GitHub Actions
- Datadog
- Grafana

Closing Thoughts

If it’s one thing I’ve learned it’s that each person develops their own methods, skills, coding style and areas of expertise over time.

So if I had one word of advice (for myself and others), STOP COMPARING YOURSELF TO OTHERS.

Also stop being so hard on yourself.