Limit the number of records in CSV output file

suel

New member
Hi - has anyone tried to limit the number of records returned when generating the csv file for export?
There is a "TimeOut" function which is ok but gives variable number of results depending on the query and the type of records reurned.
I just wondered if there was a way of setting the number of records to say something like "400" each time. e.g. the first 400 records -i.e. I can do this directly in the MYSQL table by running....... [SELECT * FROM `my table` LIMIT 0, 10] just don't know how to do it in DADABIK

Thanks
 

DebbieS

DaDaBIK Guru
In my installs I just changed the display of the export to csv option to be available when the number of records was within our server's limit and when it is not, there are short instructions advising the user the maximum records they can export - search to reduce the number of results, etc.

Here is an excerpt from my business_logic.php file:

modified code after the echo $results_table; ...

echo $results_table;

if ( $export_to_csv_feature == 1) {
if ($results_number > 4000) {
echo "<div style=\"margin:0 15px;color:#757575;\" class=\"tiny\"><i>Export to CSV feature is available only when the number of records to export is LESS than 4000. Use the search feature to reduce the number of records to view/export</i></div>";
} else {
echo " <a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=".$page."&order=".urlencode($order)."&order_type=".$order_type."&export_to_csv=1\" style=\"border:1px solid #628ABA;padding:0px 3px;\">";

txt_out ($normal_messages_ar["export_to_csv"]."</a>", "export_to_csv");
txt_out (" <a href=\"/db-csv-help.html\" title=\"Click for help with the Export to CSV feature\" onClick=\"window.open(this.href,'dbcsvhelp','width=550,height=300,toolbar=no,location=no,directories=no,status=no,menubar=yes,scrollbars=yes,copyhistory=no,resizable=yes');return false;\" class=\"question\" style=\"padding:1px 4px 2px 4px;\">?</a> ");
} // end display csv export button if record count is >= 4000
}
 

suel

New member
Hi Debbie thanks for that....however I did insert the following as per your instructions but it stops Dadabik from working, is there something wrong with the code I copied? is there something I missed out? there were a couple of smiley faces in your code and I may have taken it down wrongly -thanks...........

echo $results_table;

if ( $export_to_csv_feature == 1) {
if ($results_number > 4000) {
echo "<div style=\"margin:0 15px;color:#757575;\" class=\"tiny\"><i>Export to CSV feature is available only when the number of records to export is LESS than 4000. Use the search feature to reduce the number of records to view/export</i></div>";
} else {
echo " <a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=".$page."&order=".urlencode($order)."&order_type=".$order_type."&export_to_csv=1\" style=\"border:1px solid #628ABA;padding:0px 3px;\">";

txt_out ($normal_messages_ar["export_to_csv"]."</a>", "export_to_csv";
txt_out (" <a href=\"/db-csv-help.html\" title=\"Click for help with the Export to CSV feature\" onClick=\"window.open(this.href,'dbcsvhelp','width=550,height=300,toolbar=no,location=no,directories=no,status=no,menubar=yes,scrollbars=yes,copyhistory=no,resizable=yes');return false;\" class=\"question\" style=\"padding:1px 4px 2px 4px;\">?</a> ";
} // end display csv export button if record count is >= 4000
}
 

DebbieS

DaDaBIK Guru
ooops yes, I forgot to change those - I always forget that putting the code into regular text will turn certain spots into smilies. And I just noticed I gave you the wrong file name - oy too long of a day ...

Here it is as code - and it should go into index.php (about 1/2 way through - not the one inside case "insert" ):

[pre]
$results_table = build_results_table($fields_labels_ar, $table_name, $res_records, $results_type, "", "", $action, $where_clause_to_pass, $page, $order, $order_type);

echo $results_table;

if ( $export_to_csv_feature == 1) {
if ($results_number > 4000) {
echo "<div style=\"margin:0 15px;color:#757575;\" class=\"tiny\"><i>Export to CSV feature is available only when the number of records to export is LESS than 4000. Use the search feature to reduce the number of records to view/export</i></div>";
} else {
echo " <a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=".$page."&order=".urlencode($order)."&order_type=".$order_type."&export_to_csv=1\" style=\"border:1px solid #628ABA;padding:0px 3px;\">";

txt_out ($normal_messages_ar["export_to_csv"]."</a>", "export_to_csv" );
txt_out (" <a href=\"/db-csv-help.html\" title=\"Click for help with the Export to CSV feature\" onClick=\"window.open(this.href,'dbcsvhelp','width=550,height=300,toolbar=no,location=no,directories=no,status=no,menubar=yes,scrollbars=yes,copyhistory=no,resizable=yes');return false;\" class=\"question\" style=\"padding:1px 4px 2px 4px;\">?</a> " );
} // end display csv export button if record count is >= 4000
}
[/pre]

Huh, pasting as code didn't get rid of the smilies - still had to add spaces to get it to display right.
Hopefully this works -- remember to remove the code you pasted into business_logic.php.

Deb
 

suel

New member
Debbie - you are an angel - that worked a treat!

Can I make a small donation to the Christmas fund?

Let me know how I can do this as you have helped big time here.
 

DebbieS

DaDaBIK Guru
No problem - glad to help!
To support the program, go to http://www.dadabik.org/index.php?function=show_download.
Thanks!
 

DebbieS

DaDaBIK Guru
I know this is an old post, but I did figure out a way to allow the user to export all the results in a large database in "blocks" using LIMIT - may or may not apply to MySQL only - I only work with MySQL and Oracle. On our server, the number of records that I've capped my block at is 4000 - if this hack is used, you will need to determine the number of records you can successfully export in each block. Here goes.

index.php ............................................................................................................................
In file index.php, look for this bit of code inside the case "search":
[pre]
if ($where_clause != ""){
$sql .= " WHERE ".$where_clause;
} // end if
[/pre]

Immediately following this bit, add the following code:
[pre]
// following gets the limit values from the url which are sent via the block links
if (isset($_GET["csvstart"])) {
$csvstart = $_GET["csvstart"];
} else {
$csvstart = '';
}
if (isset($_GET["csvnorec"])) {
$csvnorec = $_GET["csvnorec"];
} else {
$csvnorec = '';
}
if ((isset($export_to_csv) && $export_to_csv == 1 && $export_to_csv_feature == 1) && ($csvstart != '' && $csvnorec != '')) {
$sql .= " LIMIT ".$csvstart.", ".$csvnorec;
}
[/pre]

Next, look for the spot where it writes the Export to CSV link under the results table:
[pre]
if ( $export_to_csv_feature == 1) {
echo "<a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=".$page."&order=".urlencode($order)."&order_type=".$order_type."&export_to_csv=1\">";

txt_out ($normal_messages_ar["export_to_csv"]."</a>", "export_to_csv");

echo "</a>";
}
[/pre]

and replace that with this (parts added shown in bold):
[pre]
if ( $export_to_csv_feature == 1) {
$csvincrement = 4000;
if ($results_number > $csvincrement) {
echo "<b style=\"color:#555555;font-weight:normal;\">Export to CSV in sections » </b>";
$csvlinks = build_csv_links($results_number, $action, $where_clause_to_pass, $fields_labels_ar);
echo $csvlinks;
} else {

echo "<a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=".$page."&order=".urlencode($order)."&order_type=".$order_type."&export_to_csv=1\">";

txt_out ($normal_messages_ar["export_to_csv"], "export_to_csv");
echo '</a>';
} // end display csv export button if record count is >= 4000
}
[/pre]

business_logic.php .............................................................................................................
In business_logic.php, the "build_csv_links" function called in index.php needs to be added. I added mine immediately following the "build_csv" function so they are together in the file.
[pre]
function build_csv_links($results_number, $action, $where_clause_to_pass, $fields_labels_ar)
// build a set of csv export links, starting from a recordset and number of records setting
// input: $res_record, the recordset, $fields_labels_ar
{
global $table_name, $function, $order, $order_type, $csv_separator, $alias_prefix, $csvincrement; // increment set in index.php
$csvlinkno = ceil($results_number / $csvincrement); // get a whole number of blocks required (rounded up to get all)

for ($cl=0; $cl < $csvlinkno; $cl++) {
$csvdispno = $cl + 1;
$csvadd[$cl] = $cl * $csvincrement;
$csvadd1[$cl] = $csvadd[$cl-1] + $csvincrement;

if ($csvadd[$cl] == 0) {
$where_start = $cl;
} else {
$where_start = $csvadd1[$cl];
}

$csvlinkdisp .= " <a href=\"".$action."?table_name=". urlencode($table_name)."&function=".$function."&where_clause=".urlencode($where_clause_to_pass)."&page=0&order=".urlencode($order)."&order_type=".$order_type."&csvstart=".$where_start."&csvnorec=".$csvincrement."&export_to_csv=1\" title=\"Section ".$csvdispno." Export\" style=\"border:1px solid #628ABA;padding:0 3px 1px 3px;\"> ".$csvdispno." </a>";
} // csvstart is the record to start with ... csvnorec is the number of records to get

if ($csvlinkno > 8) { // set here to put the export links on a new line (comment out if not required
$csvlinkdisp = '<div style="margin-top:7px;">'.$csvlinkdisp.'</div>';
return $csvlinkdisp;
} else {
$csvlinkdisp = $csvlinkdisp;
return $csvlinkdisp;
}

} // end function build_csv_links
[/pre]

Hope this helps someone out there - or if someone has found a better way, post back!
Note I've removed some of the indents in the pasted code for readibility.
 
Top