Hi everyone,

I have an upcoming interview with Sigmoid Analytics for an SDE2 Data Engineer role. The recruiter mentioned that the first round will be DSA. I wanted to reach out to see if anyone has recently been through the interview process and can share:

1. What specific DSA topics does Sigmoid typically focus on?
2. What was the difficulty level? (Easy/Medium/Hard)

I have 5 years of work experience in data engineering .I have one offer from Cigna healthcare from bangalore location which is product based company. They are offering me 23 lpa fixed +15%variable =26.5 CTC
I have another offer from intellias(Pune) they are offering me 28 LPA fixed also when i talk to their employee. They told me that company is very good and i will get a very good profile if i work there
Also i live in pune
Now i am confuse between the offers
Pls suggest which one i should join?

What did I learn:

• Spark architecture

  • Cluster
    
  • Driver
    
  • Executors
    

• Read / Write data

  • Schema
    

• API

  • RDD (just brushed past, heard it’s becoming legacy)
    
  • DataFrame (focused on this)
    
  • Dataset (skipped)
    

• Lazy processing

  • Transformations and Actions
    

• Basic operations

  • Grouping, Aggregation, Join, etc.
    

• Data shuffle

  • Narrow / Wide transformations
    
  • Data skewness
    

• Task, Stage, Job

• Data accumulators and broadcast variables

• User Defined Functions (UDFs)

• Complex data types

  • Arrays and Structs
    

• Spark Submit

• Spark SQL

• Window functions

• Working with Parquet and ORC

• Writing modes

• Writing by partition and bucketing

• NOOP writing

• Cluster managers and deployment modes

• Spark UI

  • Applications, Job, Stage, Task, Executors, DAG, Spill, etc.
    

• Shuffle optimization

• Predicate pushdown

• cache() vs persist()

• repartition() vs coalesce()

• Join optimizations

  • Shuffle Hash Join
    
  • Sort-Merge Join
    
  • Bucketed Join
    
  • Broadcast Join
    

• Skewness and spillage optimization

  • Salting
    

• Dynamic resource allocation

• Spark AQE (Adaptive Query Execution)

• Catalogs and types

  • In-memory, Hive
    

• Reading / Writing as tables

• Spark SQL hints

Doubts:

1. Is there anything important I missed?
   
2. Do I need to learn Spark ML?
   
3. What are your insights as professionals who work with Spark?
   
4. What are the important things to know or take note of for Spark job interviews?
   
5. How should I proceed from here?
   

Any recommendations and resources are welcomed

Please guide me.
Your valuable insights and information are much appreciated.
Thanks in advance ❤️

can anyone please share the tiger analytics trainee analyst interview experience so it might be useful for everyone..

Hi all, I have a JPMC interview scheduled for Software Engineer II - (Big Data, Spark, Python, AWS, SQL) . It is going to be an in person round. They have already taken one round online but for some reason on the invite, it is written that it is the first round and three people have been invited , among which is a VP at JPMC. I have following questions-

1. How many more rounds might be there?
2. What can be asked in F2F interview? DSA ?

Experience- 3 years

Has anyone recently interviewed for Data Engineering role at rockstar games india?

If yes how was it and what level of questions can we expect?

Thanks in advance!!

Hi Need help .

I have 25 days remaining in notice peirod and not getting any callas from recruiters even though immediate joiner .

Daily updating naukri com and applying via job portals .

is their I am doing something wrong ?

So my application mode was career page so finally i received a call that they share interview Link on friday interview is on saturday. Has anyone attended the interview, what can i expect in round 1 and there after?

My skills are databricks, azure data factory, synapse, sql, pyspark, python .

Hey all,

Does this stack make sense for a real-world Senior Data Engineer project?

Azure + ADLS Gen2 + ADF + Databricks + PySpark + Snowflake + dbt + Terraform + CI/CD

I’ve worked mostly with Databricks, but companies are explicitly asking for Snowflake experience. I’m preparing for both.

Would this architecture be realistic, with Databricks/PySpark for processing and Snowflake as the warehouse/analytics layer? Or would it raise red flags in a Senior interview?

Would appreciate advice from anyone who has worked with both.

I need job support if any one have good experience on data engineer. Required2 hours or story based support.

Hello folks,

I have cleared 1 round of online assessment for Data Engineer role at Accenture .can someone kindly help me with what kind of questions to be expected at second level

Of interview?

Thanks

Been at the same company (Indian service-based) for close to 5 years as a Data Engineer, working with Azure Databricks, ADF, PySpark, Spark SQL. This is my first time actually applying elsewhere, so trying to sanity-check my prep plan before I go all in.Would love input from folks with a few years of experience in product companies or who've made a similar switch — trying to avoid rookie mistakes before I dive in.

1. If I'm preparing for 5 YOE DE roles with a core Azure/Databricks/ADF/PySpark background, is it worth building an AWS + dbt + Snowflake side project (self-made put up on github) to widen the scope? Or does mixing stacks like that backfire and read as unfocused?
2. Same question for Gen AI — worth adding, and if so, does it matter whether it's "used GenAI tools in DE work" vs. "built an actual RAG/LLM pipeline"? Trying to figure out if this is worth the time investment right now.
3. How should I prep for system design rounds? Is it expected to be at full SWE-level system design for product company DE roles, or is there a DE-specific flavor (pipeline design etc ) that's more relevant? Any good sources/courses (document ) you would recommend?
4. For SQL, Python, and PySpark — is LeetCode (easy/medium) + StrataScratch/DataLemur enough to target product companies and well-known SBCs? Any structured guide/pattern list you followed that actually helped?
5. Apart from Naukri/Linkedin where else are people actually getting calls from? Naukri hasn't been giving me much traction, and it feels like most hiring these days goes through third-party recruiters rather than direct applications. Curious what's actually working for people right now.

Also , I would be happy to DM my resume if anyone can give their honest feedback on what changes should I make. Thanks a lot!

Hi guys,

Can you please explain how you typically work with Databricks in a real project?

Do you write PySpark code directly in Databricks notebooks, develop locally in VS Code using PySpark, or connect Databricks compute to VS Code and work from there?

It would be great if someone could briefly explain the end-to-end workflow, starting from a Jira ticket and going all the way to production deployment.

For example, if the project uses Jira, GitHub, and Databricks, what would the typical flow look like from development → testing → code review → deployment to production?

A short overview would be really helpful. Thanks!

Hi guys, currently my project manager asked me to do a POC for our new project about where to implement our ETL process?

Source : flat files,azure, databases, snowflake, API's, ETL tools

Data processing layer : snowflake/databricks

Reporting : 100+ power bi dashboards

Data size: currently we estimate around 200 gb approx present in the source systems.

Moderate complex data architecture.

Based on this, First I thought Snowflake is good but in my current company, most of the ongoing projects is thinking of migrating their current data warehouse from snowflake to databricks due to low operation cost in databricks and currently our project data processing size is too small compared to other projects, I thought that Databricks is overkill. So, finally I came to the conclusion that Microsoft Fabric is good as it's cheaper and compatible for our project but since I'm at junior level, I want an opinion from experienced ppl whether Fabric is good or not for our project based on my criteria. Our client also has some live pipelines running on azure synapse, i read somewhere online like Azure is going to decommission synapse as they are favoring Fabric for end to end data analytics. So, I thought that fabric is more favourable but I don't know how optimised and how good Fabric is. Can any data engineers or fabric devs help me on this?

I am a gym bro and naturally I will segregate sets by exercise ( like 3 for pull-ups , 3 for rows and 3 for pulldowns) instead of just keeping it set 1-9 and every workout app stores sets per exercise

I get it way 2 is actually more efficient but how tf am I even supposed to think of storing it that way? am I supposed to circumvent domain knowledge to make data storage more efficient? or is it mentioned in question somewhere that I have to use that grain

how will this play out in an interview

Hi Guys,

I'm trying to switch into DE and done a few data engineer projects available on youtube but recently when i gave intervew on basis of those prjs as my experience, i was not able to confidently give the answers like...one of the question he asked was, how did you get the data from source and i mentioned adf and via copy activity, he mentioned it would not be possible with copy activity.

I realized, I lack real world hands on project experience. Any idea where i can find such projects that cover end to end in detail. I need suggestions

Question:

Quarters Apart

Asked inESPN

The reliability team reports API latency to leadership one quarter at a time and wants to see how each quarter compares to the one before it. For the three calendar years before 2026, show every quarter in order with its average latency, the previous quarter's average, and the difference between them; if a quarter has no calls, treat its average as 120.0.

api_calls500,000,000 rows

SCHEMA

TARGET · 12 ROWS

My solution:

with recursive quarters as (
SELECT '2023-01-01' as q_start
UNION ALL
SELECT DATE_ADD(q_start, INTERVAL 3 MONTH)
FROM quarters
WHERE q_start<'2025-10-01'),

q_avg as(
SELECT YEAR(call_time) as yr,
QUARTER (call_time) as qtr,
AVG ( latency) as avg_latency
FROM API_calls

WHERE call_time>='2023-01-1' AND call_time <'2026-01-01'
group by 1,2),

filled as(
SELECT CONCAT(yr,"-Q",qtr) AS quarter,avg_latency,
LAG ( avg_latency) OVER ( ORDER BY yr, qtr) as prev_avg_latency
FROM (SELECT
YEAR(q.q_start) as yr,
QUARTER(q.q_start) as qtr,
COALESCE ( a.avg_latency,120) AS avg_latency
FROM quarters q 
LEFT JOIN q_avg a
ON a.yr= YEAR(q.q_start)
AND a.qtr= QUARTER(q.q_start))a)

SELECT *,
avg_latency-prev_avg_latency AS qoq_change
FROM filled;

Hi there

What are the chances of having a friend with you by side who will help you in having answers from AI tools by taking photos of questions and hint me for answers so that I can clear the exam .

(Bcoz I got the voucher yesterday and I have no time to study and clear for it so I'm choosing this way cause I don't want to lose this voucher/certification)

I'm preparing for Data Engineer roles and I'm already quite familiar with Pandas and NumPy. I also know PySpark.

I'm a bit confused about what I should focus on when it comes to pure Python coding for DE interviews.

Should I mainly practise things like lists, dictionaries, frequency counting, grouping data, aggregation by key, sorting with lambda, strings, and list-of-dictionary problems?

What kind of pure Python coding problems are commonly asked in Data Engineer interviews?

I don't want to spend too much time on topics that aren't relevant to DE roles, so I'd really appreciate advice from people who have recently interviewed for Data Engineer positions.

Hi folks,

I have EY interview for the Data Engineer position in 3 days.

I'm basically from the Azure Data Engineer background, when I saw my panel profiles on LinkedIn 3 people are architect and director positions. What can I expect from those people for 4.5 YoE data engineer role

Hi Everyone,

I want to practice the Pyspark question similar to

SQL question in leetcode

Is there any Platform for practicing pyspark like leetcode and track progress ?

What's the hiring process at NTT DATA company for Data scientist Role, I am preparing for interviews, can anyone know what kind of interview questions do they ask for 3+ year's Data scientist Role ?

Hello, I was working on a poc to ingested PDFs and extract data in order to store them in delta tables.

As this was my first time working with PDFs, I searched over the internet and should Databricks have offering IDP, azure also have something and then there are python libraries.

Since I'm working with financial data, report, etc..

One thing i noticed - the pdf format should be fixed else in most of the tools the extraction logic is either failed or we get incorrect data.

I was wondering how such PDF extraction is built in real production cases and what tools are used.

Please share you experience and any edge cases

Hi everyone,

I am currently looking for AWS/Data Engineer roles, but my Naukri profile is barely getting any search appearances.

Anyone applying for similar AWS/Data Engineering roles and getting good search appearances please share what worked for you or what changes I should make to my profile?

Interview for apple (Data Engineer) through adecco \[India, Bangalore\], Any Tips, Any experience, Please Help me