Error 429

Hello

Since I cannot update from 8.2 to 10.2 because my provider one.com doesn't support IonCube, I tried Vimexx.nl which does support this.
But here my application get the error message 429: Too many requests.
It took a while but now I know why:

Vimexx.nl uses a load balancing system (HAProxy Load Balancer) so they can counter DDoS or Brute Force attacks. My application originally had result pages of 100 records and didn't run at all. Now I lowered this to 40 records but it still doesn't run properly.

What can I do about this?
Trying yet another provider (which one??) is an option, but I think all providers will use such a load balancer in the near future.

Greetings, Ronny


You are using DaDaBIK version 8.2-Lerici enterprise, installed on 02/02/2018
PHP Version: 7.4.9
mysql version: 5.5.5-10.3.23-MariaDB-1:10.3.23+maria~bionic
Web server: Apache
Client: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:79.0) Gecko/20100101 Firefox/79.0
 

eugenio

Administrator
Staff member
Hello Ronny,
as I said in the other post you made, I don't think these "many requests" come from dadabik, at least a standard dadabik installation, have you added/modified code?

I paste here what I already replied in the other post:

-----
Hello Ronny,
this doesn't sound as a DaDaBIK- related error; dadabik doesn't run so many processes, you shouldn't get that error unless you have a huge amount of users who are using your application at the same time ... and anyway I have never seen this error on a dadabik application so far.

Can you post here the URL (you can remove the domain of the site) of a page that produces the error?
If you get it, for example, when you open a search form, I can guarantee there isn'y any special call that dadabik does here, so I would ask your hosting provider to clarify what is really happening.
-----

Again, you should ask your hosting provider to clarify, to show you the logs of the access (you can probably look at the access log yourself), that's the only way to understand what is happening.

The number of records doesn't have any impact and others hosting providers use load balancing systems but I've never seen this problem before.

Best,
 
Hello Eugenio

1. I did not change anything in the code

2. The only application that is using resources of the provider is DaDaBik.
It is a test environment, so I'm the only user.
I can provoke the error by doing things in DaDaBik.
So I have a strong feeling that the error is DaDaBik related

3. This is a part of the full log of events. In bold are the table name and column name of my application

2a02:1811:d0b:8f00:9cfd:e071:7642:4aa3 - - [18/Aug/2020:17:54:45 +0200] "GET /DDB/index.php?r=1597766085&
function=show_file&file_type=image_file&tablename=OM_voorwerpen_aanwezig&tablename_original=OM_voorwerpen_aanwezig&
file_field_name=foto_1&master_table_name=&is_items_table=0&where_field=volgnummer&where_value=38 HTTP/1.1" 508 452
"https://vommu.be/DDB/index.php?function=search&tablename=OM_voorwerpen_aanwezig" "Mozilla/5.0 (Windows NT 10.0; Win64;
x64; rv:79.0) Gecko/20100101 Firefox/79.0"

4. The application is Inventaris
user = staf, pw = 123staf
You can only read.
Records per pagina is set to 40, you can change it to 100

Greetings, Ronny
 

eugenio

Administrator
Staff member
Ok, now I understand the problem, it is related to images.

You are not seeing error 429 on the page, the page, at least the first times you try to load it, loads without errors (at least this is what happens to me with the table Voorwerpen).
You are getting error 429 on some of the images that you are loading in the results grid, so you see a "broken" picture.
DaDaBIK needs to do a call to get each of the images you need to display, there isn't much we can do about it.

After you have run some pages, you might get error 429 on the page itself because of all the failed attempts you made before. Then, after some seconds, you can run it again (this is what is happening to me)

I think you could ask to your hosting provider to relax the limits since you are calling those URLs just to show pictures!

If you hide the pictures from the results grid (and keep them in the edit and details page) I am pretty sure the problem will disappear; another workaround is to limit the record per page to 20.

Best,
 
Hello

I would like to come back to this issue.

If I understand it correctly the following is happening in order to display the report:

  • step 1: a number of records (p.e. 50) is retrieved from the DB, the data is stored in a matrix and displayed on the screen
  • step 2: for each column containing images the name of this image in each record is retrieved from the DB, the image itself is fetched from /uploads and displayed on the screen in the right place
[I know that things are more complicated then that]

Now I don't understand why in step 2 the program has to address the DB again since the names of the images are already known from step 1 and are available in the matrix. If the names were retrieved from the matrix (in stead of the DB) my problem would be solved and the speed would increment a lot.

But maybe I'm missing a point.

Greetings, Ronny
 

eugenio

Administrator
Staff member
Hello,
things are more complicated than that and I don't think the problem is related to db access but to the fact that you are calling a PHP page when you want to display a picture, that's a security measure but I think your provider doesn't like the fact your browser is calling many php pages within a limited amount of time.

Best,
 
Hello
I understand better what's happening now.
The only problem is that I don't call anything (...the fact that you are calling a php page when ...). I just use your program as it is without any changes or add-ons.
We do have a lot of pictures in the result page and we will have to change that. I think indeed that this is the only solution.

Greetings
Ronny
 

eugenio

Administrator
Staff member
Hello,
I understand your point, but at the moment the only way to make images available in dadabik without leaving the access to the image to everybody (including not-logged users) is calling a PHP url for each image. If I were you, I would try to explain this to your provider to check if they can relax their limits.
If you make the pictures available only for details and edit (and not for the results grid), the problem should disappear.

Best,
 
Top