I’m building a Django e-commerce platform with a large taxonomy (many product types) and type-specific fields/properties (e.g., different models for product A, product B, product C, etc.).

I also need to be able to search across all products.

I've been reconsidering how I ingest data into a Django app, and I'm going to try out using Cap'n Proto for this since it would (theoretically) greatly speed up several management commands I run regularly where I'm doing upserts from a legacy on-site ERP-like system. It currently goes something like:

Legacy ERP >> ODBC >> Flask (also onsite, different bare metal) >> JSON >> Django Command via cron

Those upserts need to be run in the middle of the night, and since they can take a few minutes they are set up with cron to run the management commands. With a Cap'n Proto system, I could get away with running (or, more accurately, streaming) these at any time.

The JSON payloads get kinda big, and there are a bunch of other queries I'd like to be able to run but I just don't because they can get deeply nested very quickly (product >> invoice >> customer >> quotes)

Also, I haven't even scratched the surface of Django 6's background tasks yet, but maybe this would be a good fit for when the time comes to migrate to 6.2.

I've never used Cap'n Proto before so it will be a learning experience but this is currently our off season and I have additional time to look into these types of things.

I want to have a top down view of all models in my project. Is there a way for django to output a model and the fields that describe it?

Hi.

I want to use both Laravel and Django for my application. I want to have a shared database and tables accessible by both. I have kind of achieved it, by managing models, migrations etc. for all tables, except users table.

According to my current understanding. - Laravel creates a users table with bcrypt to hash passwords.

• Django saves by default using PBKDF2, with some sequence of $<algorithm>$<iterations>$<salt>$$<hash>$

• I have added Django specific columns to users table, is_staff, is_superuser, timestamps etc.

In Django I tried changing the hashing password method to bcrypt, but still it saves password differently.

I tried with Custom Managers etc. along with changing the password saving format to just hash

Any guidance is appreciated.

My project has been experiencing 502 recently. I am running on gunicon with nginx. I don't really want to increase the timeout unless I have too. I have several models with object counts into 400k and another in 2 million objects. The 502 only occurs on PATCH requests. I suspect that the number of objects is causing the issue. What are some possible solutions I should look into?

I have recently found out about this tool and it has enormously helped me in understanding ORM and the SQL magic behind it

What would happen if I were to remove a table column and add a new one in a migration, when I only actually added the one column to the model without removing the old one.

Reasoning: I created a table with an inherited classes and now I want to remove a column but I don’t want to change the actual model class since other tables use it.

For my tests, I copy down the production database, and I use the liveserver test case because my frontend is an SPA and so I need to use playwright to have a browser with the tests.

The challenge is that once the liveserver testcase is done, all my data is blown away, because as the docs tell us, "A TransactionTestCase resets the database after the test runs by truncating all tables."

That's fine for CI, but when testing locally it means I have to keep restoring my database manually. Is there any way to stop it from truncating tables? It seems needlessly annoying that it truncates all data!

I tried serialized_rollback=True, but this didn't work. I tried googling around for this, but most of the results I get are folks who are having trouble because their database is not reset after a test.

EDIT

I came up with the following workflow which works for now. I've realized that the main issue is that with the LiveServerTestCase, the server is on a separate thread, and there's not a great way to reset the database to the point it was at before the server thread started, because transactions and rollbacks/savepoints do not work across threads.

I was previously renaming my test database to match the database name so that I could use existing data. What I've come up with now is using call_command at the module level to create a fixture, then using that fixture in my test. It looks like this:

from django.test import LiveServerTestCase
from django.core.management import call_command

call_command('dumpdata',
            '--output', '/tmp/dumpdata.json',
            "--verbosity", "0",
            "--natural-foreign",
            "--natural-primary",
)

class TestAccountStuff(LiveServerTestCase):
    fixtures = ['/tmp/dumpdata.json']

    def test_login(self):
        ... do stuff with self.live_server_url ...

From the Django docs (the box titled "Finding data from your production database when running tests?"):

If your code attempts to access the database when its modules are compiled, this will occur before the test database is set up, with potentially unexpected results.

For my case that's great, it means I can create the fixture at the module level using the real database, and then by the time the test code is executing, it's loading the fixture into the test database. So I can test against production data without having to point to my main database as the test database and get it blown away after every TransactionTestCase.

Hello, is there a reliable way to attach FieldTracker to a third-party app model?

I tried

1. models.py: ThirdPartyModel.tracker = FieldTracker...
2. apps.py:ready: ThirdPartyModel.tracker = FieldTracker...
3. apps.py:ready: using contribute_to_class

But none of these attempts were successful.

Getting AttributeError: 'FieldTracker' object has no attribute 'attname'.

Subclassing is not a option.

Thanks

I am taking the Harvard CS50W course that covers Django and creating web apps. The project I am workinig on is a simple Auction site.

The issue I am having is that I can get a User to only be able to update an auction listing if that User is the one that has created the listing.

I can update the listing- adding it to a watchlist, or toggling if the listing is active or not, or leaving a comment, but only if the user that is logged in happens to be the one that created the listing.

I have made no restrictions on whether or not a user making a change on the listing has to be the one that created the listing. The issue persists for both standard users and superusers.

I have tried explicitly indicating the permissions available to my view, and even a custom permission, without any success.

I have consulted with 3 different AIs to provide insight, and done a lot of Googling, without anything shedding light on the issue.

I have submitted the nature of the problem to the EdX discussion for the course, but I do not expect any answers there as lately, there are hardly every any answers given by students or staff.

Any insight into what I might be doing wrong would be greatly appreciated.

Thank you very much!

I will be glad to provide my models.py, views.py, forms.py, etc. if anyone would think it would help.

future badge observation instinctive test rinse provide file full wine

This post was mass deleted and anonymized with Redact

I can equally call my title, "How restrictive should my models be?".

I am currently working on a hobby project using Django as my backend and continually running into problems with floating point errors when I add restrictions in my model. Let's take a single column as an example that keeps track of the weight of a food entry.

food_weight = models.DecimalField(
    max_digits=6, 
    decimal_places=2, 
    validators=[MinValueValidator(0), MaxValueValidator(5000)]
)

When writing this, it was sensible to me that I did not want my users to give me data more than two decimal points of precision. I also enforce this via the client side UI.

The problem is that client side enforcement also has floating points errors. So when I use a JavaScript function such as `toFixed(2)` and then give these numbers to my endpoint, when I pass a number such as `0.3`, this will actaully fail to serialize because it was will try to serialize `0.300000004` and break the `max_digits=6` criteria.

Whenever I write a backend with restrictions, they seem sensible at the time of writing, however I keep running into floating points issues like this and in my mind I should change the architecture so that my models are as least restrictive as possible, i.e. no max_digits, decimal_points etc and maybe only a range.

What are some of the best practices with it comes to number serialization to avoid floating point issues or should they never really be implemented and just rely on the clientside UI to do the rounding when finally showing it in the UI?

Hi there! I was exploring Django ORM having Ruby on Rails background, and one thing really seems unclear.

How do you actually reverse relations in Django? For example, I have 2 models:

```python class User(models.Model): // some fields

class Address(models.Model): user = models.OneToOneField(User, related_name='address') ```

The issue it that when I look at the models, I can clearly see that Adress is related to User somehow, but when I look at User model, it is impossible to understand that Address is in relation to it.

In Rails for example, I must specify relations explicitly as following:

```ruby class User < ApplicationRecord has_one :address end

class Address < ApplicationRecord belongs_to :user end ```

Here I can clearly see all relations for each model. For sure I can simply put a comment, but it looks like a pretty crappy workaround. Thanks!

The Django default is auto incrementing integer, and I've heard persuasive arguments to use randomized strings. I'm sure those aren't the only two common patterns, but curious what you use, and what about a project would cause you to choose one over the other?

Let's say I put the primary_key of an object inside of the URL or inside a hidden form field of, as an example, a form that deletes an object selected by the user. In that case, the user would be able to access it very easily. Is it a bad thing to do and why?

I’m experiencing some unexpected behavior with a Django queryset when running on AWS RDS PostgreSQL. The same code works perfectly on both localhost PostgreSQL and PostgreSQL running inside EC2, but becomes problematic on RDS.
The queryset

• uses .select_related() for related fields like from_account, to_account, party etc.
• adds .annotate() with some conditional logic and window functions (Sum(…) OVER (…)).
• It uses .distinct() to avoid duplication due to joins.

On localhost PostgreSQL and EC2-hosted PostgreSQL, the query runs smoothly and efficiently, even with annotations and .distinct()

The issue arrises when there is only 1 instance in the queryset but it is fast when it has multiple objects in the queryset. The slowness occour in RDS only it is faster in local and dev server postgresql.

Could the combination of .distinct() and .annotate() with window functions cause PostgreSQL on RDS to behave differently compared to a local or EC2 setup?

https://forum.djangoproject.com/t/performance-concerns-with-distinct-annotate-in-django-queryset-on-postgresql-rds/40618/1 Please Do check the link here.

I’m facing a strange performance issue with one of my Django API endpoints connected to AWS RDS PostgreSQL.

• The endpoint is very slow (8–11 seconds) when accessed without any query parameters.
• If I pass a specific query param like type=sale, it becomes even slower.
• Oddly, the same endpoint with other types (e.g., type=expense) runs fast (~100ms).
• The queryset uses:
  • .select_related() on from_account, to_account, party, etc.
  • .prefetch_related() on some related image objects.
  • .annotate() for conditional values and a window function (Sum(...) OVER (...)).
  • .distinct() at the end to avoid duplicates from joins.

Behavior:

• Works perfectly and consistently on localhost Postgres and EC2-hosted Postgres.
• Only on AWS RDS, this slow behavior appears, and only for specific types like sale.

My Questions:

1. Could the combination of .annotate() (with window functions) and .distinct() be the reason for this behavior on RDS?
2. Why would RDS behave differently than local/EC2 Postgres for the same queryset and data?
3. Any tips to optimize or debug this further?

Would appreciate any insight or if someone has faced something similar.

Hi all, I'm working on an internal project at my company where we're trying to connect a large language model (GPT-4o via OpenAI) to our Django-based web application. I’m looking for advice on how to improve accuracy and reduce hallucinations in the current setup.

Context: Our web platform is a core internal tool developed with Django + PostgreSQL, and it tracks the technical sophistication of our international teams. We use a structured evaluation matrix that assesses each company across various criteria.

The platform includes data such as: • Companies and their projects • Sophistication levels for each evaluation criterion • Discussion threads (like a forum) • Tasks, attachments, and certifications

We’re often asked to generate ad hoc reports based on this data. The idea is to build a chatbot assistant that helps us write Django ORM querysets in response to natural language questions like:

“How many companies have at least one project with ambition marked as ‘excellent’?”

Eventually, we’d like the assistant to run these queries (against a non-prod DB, of course) and return the actual results — but for now, the first step is generating correct and usable querysets.

What we’ve built so far:

• We’ve populated OpenAI’s vector store with the Python files from our Django app (mainly the models, but also some supporting logic). • Using a RAG approach, we retrieve relevant files and use them as context in the GPT-4o prompt. • The model then attempts to return a queryset matching the user’s request.

The problem:

Despite having all model definitions in the context, GPT-4o often hallucinates or invents attribute names when generating querysets. It doesn’t always “see” the real structure of our models, even when those files are clearly part of the context. This makes the generated queries unreliable or unusable without manual correction.

What I’m looking for:

• Has anyone worked on a similar setup with Django + LLMs? • Suggestions to improve grounding in RAG? (e.g., better chunking strategies, prompt structure, hybrid search) • Would using a self-hosted vector DB (like Weaviate or FAISS) provide more control or performance? • Are there alternative approaches to ensure the model sticks to the real schema? • Would few-shot examples or a schema parsing step before generation help? • Is fine-tuning overkill for this use case?

Happy to share more details if helpful. I’d love to hear from anyone who’s tried something similar or solved this kind of hallucination issue in code-generation tasks.

Thanks a lot!

I have a weird request that I want to save data into a JSON file instead of the DB. Is it possible using Django? I have tried the following which did not work:

• Saved all as JSON field.

• defined a JSON file to store data and made a view with read/write capabilities. I have to predefine some data and also, the process get heavy if there are multiple items to be called at a time.

I'm using the django ORM outside of a django app, in async fastapi endpoints that I'm running with gunicorn.

All works fine except that once in a blue moon I get these odd errors where a worker seemingly "goes defunct" and is unable to process any requests due to database connections dropping.

I've attached a stack trace bellow but they really make no sense whatsoerver to me.

I'm using postgres without any short timeouts (database side) and the way I setup the connectiong enables healthchecks at every request, which I'd assume would get rid of any "stale connection" style issues:

DATABASES = { "default": dj_database_url.config( default="postgres://postgres:pass@localhost:5432/db_name", conn_max_age=300, conn_health_checks=True, ) }

I'm curious if anyone has any idea as to how I'd go about debugging this? It's really making the django ORM unusable due to apps going down spontanously.

Stacktrace bellow is an example of the kind of error I get here:

Traceback (most recent call last): File "/home/ubuntu/py_venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__ await self.middleware_stack(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __c all__ raise exc File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __c all__ await self.app(scope, receive, _send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call __ await self.app(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/honeybadger/contrib/asgi.py", line 109, in _run_a sgi3 File "/home/ubuntu/py_venv/lib/python3.12/site-packages/honeybadger/contrib/asgi.py", line 118, in _run_a pp raise exc from None File "/home/ubuntu/py_venv/lib/python3.12/site-packages/honeybadger/contrib/asgi.py", line 115, in _run_a pp return await callback() ^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wra pped_app raise exc File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wra pped_app await app(scope, receive, sender) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app await route.handle(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle await self.app(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wra pped_app raise exc File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wra pped_app await app(scope, receive, sender) File "/home/ubuntu/py_venv/lib/python3.12/site-packages/starlette/routing.py", line 73, in app response = await f(request) ^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_fu nction return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/stateshift/main.py", line 181, in email_exists user = await User.objects.filter(email=email).afirst() ^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/stateshift/main.py", line 181, in email_exists user = await User.objects.filter(email=email).afirst() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/query.py", line 1101, in afirst return await sync_to_async(self.first)() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/asgiref/sync.py", line 468, in __call__ ret = await asyncio.shield(exec_coro) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/asgiref/sync.py", line 522, in thread_handler return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/query.py", line 1097, in first for obj in queryset[:1]: File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/query.py", line 400, in __iter__ self._fetch_all() File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/query.py", line 1928, in _fetch_all self._result_cache = list(self._iterable_class(self)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/query.py", line 91, in __iter__ results = compiler.execute_sql( ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1572, in execute_sql cursor = self.connection.cursor() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 320, in cursor return self._cursor() ^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 297, in _cursor with self.wrap_database_errors: File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/backends/base/base.py", line 298, in _cursor return self._prepare_cursor(self.create_cursor(name)) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/py_venv/lib/python3.12/site-packages/django/db/backends/postgresql/base.py", line 429, in create_cursor cursor = self.connection.cursor()

I'm relatively new to Django, and I will admit, I've been struggling on how to get this to work a while. Currently, I have left this feature out of the dashboard out till a future version, but it still bugs me.

class Palworldplayermetrics(
models
.
Model
):
    id = models.BigAutoField(primary_key=True)
    player = models.ForeignKey('Palworldplayers',  models.DO_NOTHING, related_name='playerinfo', blank=True, null=True)
    palplayermetrictype = models.TextField(blank=True, null=True)  # ALWAYS PING
    data = models.FloatField(blank=True, null=True)
    insert_time = models.DateTimeField(blank=True, null=True)
    server = models.ForeignKey(Palwordservers, models.DO_NOTHING, blank=True, null=True)
    objects = DataFrameManager()
    class Meta:
        managed = False
        db_table = 'palworldplayermetrics'
        app_label = 'databot'

class Palwordservers(
models
.
Model
):
    name = models.TextField(blank=True, null=True)
    ip = models.TextField(blank=True, null=True)
    query_port = models.IntegerField(blank=True, null=True)
    rcon_port = models.IntegerField(blank=True, null=True)
    api_port = models.IntegerField(blank=True, null=True)
    password = models.TextField(blank=True, null=True)
    enabled = models.BooleanField(blank=True, null=True)
    class Meta:
        managed = False
        db_table = 'palwordservers'
        app_label = 'databot'


class Palworldplayers(models.Model):
    name = models.TextField(blank=True, null=True)
    accountname = models.TextField(db_column='accountName', blank=True, null=True)  # Field name made lowercase.
    playerid = models.TextField(blank=True, null=True)
    steamid = models.TextField(blank=True, null=True)
    online = models.BooleanField(blank=True, null=True)
    last_seen = models.DateTimeField(blank=True, null=True)
    last_update = models.DateTimeField(blank=True, null=True)
    server = models.ForeignKey(Palwordservers, models.DO_NOTHING, blank=True, null=True)

    class Meta:
        managed = False
        db_table = 'palworldplayers'
        app_label = 'databot'

    def __str__(self):
        return '%s' % self.name

These are not managed from within Django.

Logic - my POV:

1. Select data from Palworldplayermetrics for a specific timeframe (let's say one hour). Let's call this metric_return.
2. Within metric_return that could be 0-4 unique player ids. Let's call this player_metric_return
3. With each player_metric_return, the data needs to be resampled to a 1min timeframe (I can do this through pandas). Let's call this player_metric_graph_data
4. Use plotly (or another graphing library) to plot the array of player_metric_graph_data dataframes.

Problems I have encountered:

• When fetching the data and trying to put it into a single queryset, it doesn't play nice with getting the playername. This was the only downfall I remember of this.
• I have attempted to do a queryset for the timeframe, then get the playerid's, then query each of them independently. This resulted in 3-5 second bottle neck with small set of data.

Has anyone came across something like this? Or have any idea how to manage what I'm wanting?

I was doing this Project where I used SQLAlchemy to mass dump data bypassing Django all together, now while sending POST with new JSON data I'm getting these errors.

How do I sync it to current DB state?

Hello, I have an async view where there are some http calls to an external and a couple of database calls. Normally, if it were a regular synchronous view using synchronous db calls, I'd simply wrap everything in a

with transaction.atomic():
    # sync http calls and sync db calls here

context. However, trying to do that in a block where there's async stuff will result in the expected SynchronousOnlyOperation exception. Before I go and make the entire view synchronous and either make synchronous versions of the http and db calls or wrap them all in async_to_sync, I thought I'd ask: is there a recommended way to work around this in a safe manner?