Mosel database and spreadsheet examples
All examples in this directory are explained in the whitepaper
'Using ODBC and other database interfaces with Mosel'.
Examples that have several versions obey the following naming convention:
'filename*.mos' where '*' =
[] - 'odbc' driver,
1 - working with text data files,
2 - ODBC connection using SQL statements,
3 - 'excel' driver (Windows only),
4 - Oracle (OCI) interface,
5 - 'xls'/'xlsx' driver,
6 - 'csv' driver
|
| Reading sparse data from text files, spreadsheets and databases, and from memory: Data input, initializations from, SQL, ODBC, spreadsheet, CSV
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
Suppose we want to read in data of the form
from a file, setting up a dynamic array A(range,
range) with just the A(i,j)= value(i,j)
for the pairs (i,j) which exist in the file. Here
is an example which shows different ways of doing this. We read data
from
- differently formatted text files (example trio.mos described in the 'Mosel User Guide', Section 3.4 Reading sparse data): using 'initializations from', 'readln', or 'diskdata',
- spreadsheets or databases (examples duo.mos, duosheet.mos, duooci.mos,
duoexc.mos, and duodd.mos described in the Xpress Whitepaper 'Using ODBC and other database interfaces with Mosel') ,
- the application (C/Java) executing the Mosel model (examples datainputc.mos and datainputj.mos
into different arrays, and using writeln
show that the arrays hold identical data.
Accessing spreadsheets and databases
With initializations from data from spreadsheets or databases is read in a similar way as text files are accessed; switching between data sources can
be done by replacing the I/O driver name and the data source in the extended filename, such as "mmsheet.excel:data.xls" to access the spreadsheet data.xls or "mmodbc.odbc:data.mdb" to read from the MS Access database data.mdb and the
string "mmodbc.odbc:DSN=mysql;DB=data" could be used to access
a mysql database named data.
Instead of using an initializations block, we may read data from a database into a Mosel model by writing out the corresponding SQL statements (this format allows
fine-tuning of the queries and
gives access to a larger range of commands, in particular when working
with databases). When developing applications with ODBC/SQL statements, the user is advised to use the following two parameter
settings to obtain error messages and debugging information from ODBC:
setparam("SQLverbose", true)
setparam("SQLdebug", true)
The module mmoci (requires a separate license) defines a software-specific interface to Oracle databases that is used in a similar way as
the ODBC connection.
Using initializations from the extended filename will take a form like "mmoci.oci:myname/mypassword@dbname" and it is also possible to work with PL/SQL statements like the following:
OCIlogon('myname/mypassword@dbname')
OCIexecute("select Index_i,Index_j,Value from MyDataTable", A5)
OCIlogoff
Data input in memory using I/O drivers
Using initializations from it is also possible to read data into a Mosel model from the calling application (C/C++, C#, Java) using a combination of
the I/O drivers mem (data held in memory) and raw driver (data in binary format) in C, drivers dotnet and dotnetraw in C# programs, and drivers java and jraw in Java programs.
Other options for data input
Other possibilities of inputting data into a Mosel model include
- Static modules: data exchange between the calling application and a model via custom functions, using the Mosel Native Interface - see the
example of embedding the cutting stock problem in the Whitepaper 'Embedding Optimization Algorithms'.
- Custom format/custom data sources: the Mosel Native Interface allows the user to implement his own I/O drivers (to be used with
initializations blocks and all other Mosel functionality
involving file access, such as matrix export or model compilation).
Possible uses of such user-written drivers include
encoding/decoding of data on the fly or compression/decompression
of data.
See the Xpress Whitepaper 'Generalized file handling in Mosel' for
the complete code and documentation of an example implementing
a compression driver with the zip library.
Furthermore, the Mosel IO
driver pipe may be used to recover the output of an external
program. See the section on I/O drivers in the 'Mosel Libraries
Reference Manual' for more information.
|
| File(s): |
duo.mos, duosheet.mos, duoexc.mos, duodd.mos, duooci.mos |
| Data file(s): |
data.csv, data.xls, data.xlsx, data.mdb, data.sqlite, datadd.csv |
|
|
| Data output to text files, spreadsheets and databases, and to memory: Data output, initializations to, SQL, ODBC, spreadsheets, CSV
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
This example shows different ways of writing out data from a Mosel model, namely to
- differently formatted text files (example trio_out.mos described in the 'Mosel User Guide', Section 10.2 File output): using 'initializations to', free-format output with 'write'/'writeln',
or 'diskdata' format,
- spreadsheets or databases (examples duo_out.mos, duosheet_out.mos, duooci_out.mos, duoexc_out.mos, and duodd_out.mos described in the Xpress Whitepaper 'Using ODBC and other database interfaces with Mosel'),
- the application (C/Java) executing the Mosel model (examples dataoutputc.mos and dataoutputj.mos
Accessing spreadsheets and databases
Using initializations to data is written to spreadsheets or databases through the ODBC connector in a similar way as text files are accessed; all
necessary SQL commands are generated automatically. With the extended filename to "mmodbc.odbc:data.mdb" the data
will be written to the MS Access database data.mdb and the
string "mmodbc.odbc:DSN=mysql;DB=data" could be used to access
a mysql database named data.
Instead of using an initializations block, we may write out data from a Mosel model directly with the corresponding SQL statements (this makes it possible, for
example, to clear the data table before writing to it - this functionality is not available with spreadsheets)
SQLconnect("data.sqlite")
SQLexecute("delete from MyOutTable")
SQLexecute(
"insert into MyOutTable (Index1,Index2,AValue) values (?,?,?)", A)
SQLdisconnect
When working with Excel spreadsheets it is recommended to use one of the dedicated spreadsheet drivers from the mmsheet module: with the excel driver, the spreadsheet may remain
open while writing to it from a Mosel model, in this case the data
written out to the spreadsheet does not get saved, making it possible
to re-run the application with different data sets without causing
any lasting modifications to your spreadsheet. All other spreadsheet drivers require the spreadsheet file to be closed.
The xls/xlsx drivers do not require an installation of Excel and are available for certain non-Windows platforms.
The module mmoci (requires a separate license) defines a software-specific interface to Oracle databases that is used in a similar way as
the ODBC connection provided through mmmodbc.
With initializations to an extended filename will take a form like "mmoci.oci:debug;myname/mypassword@dbname" and it is equally possible to use PL/SQL statements directly.
Data output in memory using I/O drivers
Using initializations to it is also possible to send data from a Mosel model in memory to the calling application (C/C++, C# or Java). This is done
using a combination of the I/O drivers mem (data held in memory) and raw driver (data in binary format) in C, drivers dotnet and dotnetraw in C# programs and drivers java and jraw in Java programs.
Other options for data output
Other possibilities of outputting data from a Mosel model include
- Use of the Mosel Libraries: information about the model objects is retrieved after the execution of the model is terminated. See the 'Mosel User Guide',
Part III Working with the Mosel Libraries and the example dataafterexec.c.
- Redirection of the output and error streams: to a file or to the calling application with the help of a callback function (I/O driver cb).
- Static modules: data exchange between the calling application and a model via custom functions, using the Mosel Native Interface. See the
example datainout.c.
- Custom formats/custom data sources: the Mosel Native Interface allows the user to implement his own I/O drivers (to be used with
initializations blocks and all other Mosel functionality
involving file access, such as matrix export or model compilation).
Possible uses of such user-written drivers include
encoding/decoding of data on the fly or compression/decompression
of data.
See the Xpress Whitepaper 'Generalized file handling in Mosel' for
the complete code and documentation of an example implementing
a compression driver with the zip library.
Furthermore, the Mosel I/O
driver pipe may be used to channel the output of a Mosel
model to an external
program. See the section on I/O drivers in the 'Mosel Libraries
Reference Manual' for more information.
|
| File(s): |
duo_out.mos, duosheet_out.mos, duoexc_out.mos, duodd_out.mos, duooci_out.mos |
| Data file(s): |
data.csv, data.xls, data.xlsx, data.mdb, data.sqlite |
|
|
| Dataframe formats: dataframe, union types, CSV format
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
This example demonstrates dataframe-style representation of data. This functionality is provided for CSV format files via
the 'diskdata' I/O driver and subroutine of module mmetc, for the spreadsheet I/O drivers 'xls', 'xlsx', and 'csv' of module mmsheet, and for databases accessed through an ODBC connection via SQL commands.
- dataframecsv.mos: reading and writing dataframes for CSV format data via 'diskdata'
- dataframedb.mos: dataframe-style reading of database tables via 'SQLdataframe'
- dataframesht.mos: reading and writing dataframes for spreadsheets and CSV format data via mmsheet
|
| File(s): |
dataframecsv.mos, dataframedb.mos, dataframesht.mos |
| Data file(s): |
dframedata.csv, dframedata.xls, dframedata.xlsx |
|
|
| Dense vs. sparse data format: array types, noindex
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
The example 'indexeg' shows how to read data tables with different
formats from various data sources:
- a dense 3x2 table
- a dense 2x3 table
- a sparse 2x3 table
- a 2x4 table with partially indexed format
A data may be accessed from Mosel through an ODBC connection (using the odbc driver, indexeg.mos, or with SQL statements, indexeg2.mos), with the software-specific driver excel (indexeg3.mos), or with the portable mmsheet module that can be used to read and write xls, xlsx (indexeg5.mos) and csv (indexeg6.mos) files.
The first two methods (odbc driver and SQL statements) apply to spreadsheets and databases. In addition, for Oracle databases we show how to use the
software-specific connection provided by the module mmoci (indexeg4.mos).
|
| File(s): |
indexeg.mos, indexeg2.mos, indexeg3.mos, indexeg4.mos, indexeg5.mos, indexeg6.mos |
| Data file(s): |
indexeg.csv, indexeg.xls, indexeg.xlsx, indexeg.sqlite |
|
|
| Auto-indexation for single-dimensional arrays: auto-indexation
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
This example shows how to read single-dimensional data tables from a spreadsheet or database file that only contains the data
values. The data arrays must have an index set of type range that gets populated automatically based on the row count, either starting with the default value 1 or with a specified start
value. Several arrays can be populated via a single statement.
|
| File(s): |
autoindex.mos |
| Data file(s): |
adt.csv, adt.xls, adt.xlsx, adt.sqlite |
|
|
| Writing out solution values to text files, spreadsheets or databases: solution data output, 'evaluation of'
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
Solution values of decision variables or constraints need to be copied into an array. The contents of this array can then
be written to a spreadsheet or database using ODBC or software-specific drivers (or simply to text file using Mosel's default
output).
- Spreadsheets and databases can be accessed from Mosel through an ODBC connection using the odbc driver (soleg.mos) or with SQL statements (soleg2.mos).
- If Microsoft Excel is installed, the software-specific driver excel (soleg3.mos) can be used.
- For Oracle databases a software-specific connection is provided by the module mmoci (soleg4.mos).
- For generic spreadsheets it is recommended to use the portable mmsheet module that provides subroutines to read and write xls and xlsx (soleg5.mos) and csv (soleg6.mos) files.
As an alternative to the explicit copying of arrays of solution values you can use the keyword evaluation in initializations to blocks.
If preceded by the marker evaluation of solution values, results of Mosel
functions or other expressions can be used directly in the initializations to block as shown in the example initeval.mos.
|
| File(s): |
soleg.mos, soleg2.mos, soleg3.mos, soleg4.mos, soleg5.mos, soleg6.mos |
| Data file(s): |
soleg.csv, soleg.mdb, soleg.xls, soleg.xlsx, soleg.sqlite |
|
|
| Spreadsheets and databases: working with multiple data tables and arrays: data input/output, multiple tables, multiple data arrays
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
A database table (or a spreadsheet range) may contain in its different fields (columns) the data for several Mosel arrays
and inversely, a Mosel array may correspond to several data tables (ranges). We have here examples of various different cases:
- using the odbc driver,
- formulated with SQL statements,
- using the excel driver,
- using the oci driver,
- using the xls driver, and
- using the csv driver.
for three sets of examples:
- Reading several arrays from a single database table/spreadsheet range (multicol.mos)
- Outputting several arrays into a single database table/spreadsheet range (multiout.mos)
- Reading an array from several tables/ranges (multitab.mos)
|
| File(s): |
multicol.mos, multicol2.mos, multicol3.mos, multicol4.mos, multicol5.mos, multicol6.mos, multiout.mos, multiout2.mos, multiout3.mos, multiout4.mos, multiout5.mos, multiout6.mos, multitab.mos, multitab2.mos, multitab3.mos, multitab4.mos, multitab5.mos, multitab6.mos |
| Data file(s): |
multicol.dat, multicdd.dat, multicol.csv, multicol.mdb, multicol.xls, multicol.xlsx, multicol.sqlite, multiout.dat, multicol.csv, multicol.mdb, multicol.xls, multicol.xlsx, multicol.sqlite, multitab.csv, multitab.dat, multitab.mdb, multitab.xls, multitab.xlsx, multitab.sqlite |
|
|
| Formulation of SQL (selection) statements: Advanced SQL functionality, retrieve database and table structure
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
If some advanced SQL functionality is required when accessing
a data source through an ODBC or OCI connection it will in general
be necessary to formulate the corresponding SQL statements
directly in the Mosel model instead of using the corresponding IO
driver in initializations blocks. We show here two examples:
- Selection statements using columns in
inverted order (ODBC: odbcinv.mos, Oracle: ociinv.mos)
- SQL with conditional selection statements and functions
(ODBC: odbcselfunc.mos, Oracle: ociselfunc.mos)
- Retrieving information about database table structure: list of database tables, list of field names, primary keys (odbcinspectdb.mos)
- Implementing 'stop on error' behaviour for SQL statements (odbcchkstatus.mos)
|
| File(s): |
odbcinv.mos, odbcselfunc.mos, odbcchkstatus.mos, ociinv.mos, ociselfunc.mos, odbcinspectdb.mos |
| Data file(s): |
multicol.mdb, multicol.sqlite, ocidata.dat, odbcsel.mdb, odbcsel.xls, odbcsel.sqlite |
|
|
| Reading and writing records and lists: record data structure, list data structure
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
This set of examples shows how to work with advanced data structures when accessing data in spreadsheets and databases:
- reading in records (recordin*.mos)
- writing out records (recordout*.mos)
- reading and writing lists (listinout*.mos)
The basic model version uses an ODBC connection to spreadsheets or databases through the odbc driver, model version (2) uses ODBC through SQL statements, and model version (3) uses the software-specific driver excel to access Excel spreadsheets. Model version (1) shows how to work with Mosel's text data file format.
|
| File(s): |
recordin.mos, recordin1.mos, recordin2.mos, recordin3.mos, recordin4.mos, recordin5.mos, recordin6.mos, recordout.mos, recordout1.mos, recordout2.mos, recordout3.mos, recordout4.mos, recordout5.mos, recordout6.mos, listinout.mos, listinout1.mos, listinout2.mos, listinout3.mos, listinout4.mos, listinout5.mos, listinout6.mos |
| Data file(s): |
recorddata.dat, recorddata.csv, recorddata2.csv, recorddata.mdb, recorddata.xls, recorddata2.xls, recordout_templ.xls, recorddata.xlsx, recorddata2.xlsx, recorddata.sqlite, listdata.dat, listdata.csv, listdata.mdb, listdata.xls, listdata.xlsx, listdata.sqlite |
|
|
| Reading and writing union types: Types any, date, time; data input/output
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
A union is a container capable of holding an object of one of a predefined set of types. Its declaration specifies the set
of compatible types or the predefined union type 'any'.
The examples display the resulting types when populating union type entities from external data sources. It is also shown
how to convert the type of union entities from textual types to 'date' or 'time' types as defined by the Mosel module mmsystem.
- The model version in unioninout1.mos shows how to work with Mosel's text data file format.
- Databases can be accessed from Mosel through an ODBC connection using the odbc driver (unioninout.mos) or with SQL statements (unioninout2.mos).
- For Oracle databases a software-specific connection is provided by the module mmoci (unioninout4.mos).
- Microsoft Excel spreadsheets can be accessed via the software-specific driver excel (unioninout3.mos). Alternatively, models can use the dedicated I/O drivers xsl/xlsx (unioninout5.mos) from the mmsheet module that support manipulating spreadsheet files on various platforms without having to install Microsoft Excel.
- Module mmsheet also provides an I/O driver for csv files (unioninout6.mos).
|
| File(s): |
unioninout.mos, unioninout1.mos, unioninout2.mos, unioninout3.mos, unioninout4.mos, unioninout5.mos, unioninout6.mos |
| Data file(s): |
uniondata.csv, uniondata2.csv, uniondatain.dat, uniondata.xls, uniondata.xlsx, uniondata.sqlite |
|
|
| Reading and writing dates and times: data types date, time, datetime
|
| |
|
| Type: |
Programming |
| Rating: |
3 (intermediate) |
| Description: |
This set of examples shows how to work with date and time data types when accessing data in spreadsheets and databases.
The basic model version uses an ODBC connection to databases through the odbc driver, model version (2) uses ODBC through SQL statements and version (4) shows the same for Oracle databases accessed via
the OCI. Model versions (3) and (5) work with different spreadsheet drivers, and version (6) with CSV data format.
|
| File(s): |
datesinout.mos, datesinout2.mos, datesinout3.mos, datesinout4.mos, datesinout5.mos |
| Data file(s): |
datetime.csv, recorddata2.csv, datetime.xls, datetime.xls |
|
|
| Reading 3-dimensional arrays: Multidimensional array input, option partndx
|
| |
|
| Type: |
Programming |
| Rating: |
2 (easy-medium) |
| Description: |
This example shows how to read a 3-dimensional array from an excel file (threedimarr.mos) and from a generic spreadsheet (threedimarr5.mos) where the last index is specifed 'across the columns',
resulting in a pivot-style table format where the first N-1 columns contain index values (here: 2 index columns). Model versions
(2) and (4) show the same example for databases, and model version (6) works with data in CSV format.
Note that standard use of the 'partndx' option for spreadsheets assumes that the last index is a set of type 'range', the
example partidxstr.mos defines a utility routine 'makelabel' that makes it possible to work with a 'set of string' for the
last index.
|
| File(s): |
threedimarr.mos, threedimarr2.mos, threedimarr3.mos, threedimarr4.mos, threedimarr5.mos, threedimarr6.mos, partidxstr.mos |
| Data file(s): |
threedim.csv, threedim.xls, threedim.xlsx, threedim.mdb, threedim.sqlite, partidxdata.csv, partidxdata.xls, partidxdata.xlsx |
|
|
|
|