Introduction
Nowadays computers are all
multi-cores. Therefore, we can use Queue, Stack, Priority Queue, List to
distribute workloads in multithreading programming.
If the workloads can be completed within a finite timeframe and
number of workloads are much bigger than the computer’s available threads,
we can tag these workloads and place the tags into Queue, Stack, Priority
Queue, Linked List. Subsequently these workloads / processes can then be
distributed accordingly among all the available threads to execute.
To tag the workloads, we can use numbering or other form of
unique, not repetitive ID.
Each threads / remote computers are equipped with the coding to
process the workloads (OOP). When a thread is read / edit the task queue /
stack / list, the thread will have exclusive access to the Task Queue.
Other threads will enter ‘WaitSleepJoin’ state after knowing Task Queue is
locked (‘isLocked’ = true). After the thread finished read / edit the Task
Queue, it will unlock the Task Queue (‘isLocked’ = false) and tell one
waiting thread (if there is one) to become ready to execute (Running
state). The waiting time is very short and negligible, because a thread
only read / edit single Tag Number from the Task Queue.
Hash Table and Array
Both Hash
Table (equivalent to Visual Studio’s Dictionary) and Array can be
used to save workloads result if we want to trace the results are coming
from which workloads.
Hash Table has memory allocation advantage over Array. Before
using Array, we need to declare Array. When declaring an Array, program
will reserve a continuous chunk of memory block. In Hash Table, single node
size memory is reserved when new node created; each node memory location
are scattered around and are not cluster together.
Array allows concurrent multithreads data read / write, because
the required memory block are reserved in advance. Hash Table can also
allow concurrent multithreads data read / write, provided Hash Table memory
allocation are created before the data editing.
If memory allocation is done upon adding a new node into Hash
Table, the thread needs to lock the Hash Table, preventing other threads
from read / write the Hash Table; after the thread added the node into Hash
Table, the thread unlocks the Hash Table, allowing other threads to read /
write data into the Hash Table.
Hash Table and Array are not suitable to store workload tags,
because the tags should be removed out upon the threads are processing the
workloads. Furthermore, new workload tags are being added into the queue /
stack / list at any time.
Queue and Stack
If
the workloads need to be processed sequentially, we can use queue and stack
to save the workloads tag.
Queue
is First In First Out (FIFO) manner; Stack
is Last In Fist Out (LIFO) manner.
Priority Queue
When the workloads have priority,
highest priority workloads must be attended first. We can use Priority
Queue (command in Visual Studio is ‘SortedList’).
Visual Studio ‘SortedList’ class
represents a collection of key-and-value pairs that are sorted by the keys
and are accessible by key and by index. Here the keys indicate individual
workloads’ priority number, value is the tag number.
Linked List
When we want to
process the workloads in random manner, we can use Linked List to save
the workloads tag.
Example
This program print-out numbers 1 to 100000 individually without
repeating in random manner using multi-threading programming.
Version ‘1’ in c#
Console App: Example_v1.cs.txt
Version ‘2’ New
single number (100001~200000) is added into the List
every millisecond while threads are printing the numbers out.
In c# Console
App: Example_v2.cs.txt
Version ‘3’: Added
Array and Hash Table into the code. Randomly generate some data and save
into Array / Hash Table. Example_v3.cs.txt
Version ‘4’: Using
C# lock
statement. The lock
statement acquires the mutual-exclusion lock for a given object,
executes a statement block, and then releases the lock. Other threads will
have to wait until the lock is released. The statement method same to
coding that using Monitor
Class above. Example_v4.cs.txt
Version ‘5’: Using Monitor
Class. The ‘try-finally’ block is used so that the ‘isLocked’ variable will always set to false when the try/catch block
leaves the execution, regardless the try block terminates normally or
terminates due to an exception. Hence allows other threads to read / edit
task queue afterwards. Example_v5.cs.txt
Deadlock / Livelock may
happen if statement block is depending on external members.
Example 2
In previous example, each subprocess threads will exit
individually when found the task queue / stack / list is empty. Main
process thread will end when all the sub-process threads are ended.
In some scenarios
we want the main program to continue running even when task
queue / stack / list is empty, waiting for new tasks to be added into the
task queue and letting subprocess threads execute the tasks.
In this example,
the subprocess threads and main program will not exit when the task
queue / stack / list is empty. Instead, worker threads will enter
‘WaitSleepJoin’ state until new workloads tag numbers are added into the
task queue / stack / list. Subprocess
worker threads will become Running state when
obtained the lock.
Producer threads will input a number of workloads tag numbers
into the task queue / stack / list. Then sleep for 5 seconds. Wake-up,
input another number of workloads tag numbers into the task queue / stack /
list. Therefore, certain time within these 5 seconds interval, the task
queue / stack / list is empty.
Version ‘1’: Although main program will not exit when task
queue / stack / list is empty. But this program also unable to exit by
itself, except force termination ctrl-c.
c# Console App: multithreaded_message_queue1.cs.txt
Version ‘2’: amended coding. press ‘q’ button to one-by-one end
all the subprocess worker threads and then main program thread.
c# Console App: multithreaded_message_queue2.cs.txt
Example 3: Multithreaded Chat Server application establish network
connections to multiple Clients concurrently using multiple ephemera ports
in C#
In this chat server & chat clients example, I am using
shared-Stack to distribute workloads of a multithreaded Chat-Server
application. The Chat-Server application should able to accept multiple
chat-clients network connections concurrently.
The example code below
demonstrates how it works.
Single multithreaded server application can maintain
communications and duplex data transmission with multiple clients
concurrently using multiple ports.
1.
Multithreaded chat server application
declares 1 thread to run ‘Portal Server’ using UDP / TCP transport layer
protocol on port 50000; four threads to run ‘Worker Server’ using TCP
transport layer protocol on port ‘50001~50004’. All threads listening to
different network ports.
2.
All chat clients who want to
establish network connection with chat server firstly go to ‘Portal Server’
on port 50000 using UDP / TCP transport layer protocol. ‘Portal Server’
will inform chat clients which available worker servers’ network port to
connect by using shared workloads-stack.
3.
Chat client establish connection with
the available ‘Worker Server’ on assigned port using TCP
transport layer protocol.
4.
If chat client did not establish
connection with the ‘Worker Server’ on assigned port within preset timeout
period, the port number will be pushed back into shared workloads-stack.
The assigned ‘Worker Server’s key will be changed too.
5.
To avoid stray chat-clients connect
to ‘Worker Server’ directly without firstly visit ‘Portal Server’.
a.
chat-client should get a key from
‘Portal Server’.
b.
Afterward, chat-client connect to the
assigned port of ‘Worker Server’ and send the key to maintain connection.
c.
If ‘Worker Server’ did not get the correct
key in the first string sent by client, ‘Worker Server’ closes the network
connection. Worker server will change to new key. Client only has one
chance sends the correct key. Client will not able to attempt second time.
Client should visit ‘Portal Server' again to re-attempt connection.
d.
After ‘Worker Server’ close a
connection, either closed normally or closed with exceptional error, or
closed because client sent a wrong key, ‘Worker Server’ will change the
key. Hence, client will not able to connect to the same ‘Worker Server’
with the previously received key, or by multiple key trial-and-error
method.
6.
After chat-client established network
connection with the assigned ‘Worker Server’. If chat-client idle and do
not send message to ‘Worker Server’ for a preset timeout period, the
connection will be closed. ‘Worker Server’ send string message
"SERVER>>> IDLE TIMEOUT" to inform chat-client about
idle connection timeout. ‘Worker Server’ push it port number back to shared
workloads-stack, ready to listen new incoming connection.
Optionally we can preset maximum
total connected time of client to ‘Worker Server’. After maximum total
connected time, ‘Worker Server’ will close the connection with client by
sending string message “SERVER>>> TOTAL TIME TIMEOUT” to client.
Even though client remain active during the period.
7.
If all ‘Worker Server’ are currently
occupied, ‘Portal Server’ send string message "SERVER>>>
FULL" to the new chat-client. New chat-client should sleep for 2
seconds before re-attempt to inquire ‘Portal Server’.
8.
Assume ‘class_currTimeDate’ is a RTC
that trigger interrupt in 1 second or 2 seconds frequency.
9.
When chat-client wants to terminate
the connection, it sends string message "CLIENT>>>
TERMINATE" to chat-server.
In the below simulated chat server-client applications example,
via
chat server application, port number ‘50001’ chat-client will
chat with port number ‘50002’ chat-client; port number ‘50003’ chat-client
will chat with port number ‘50004’ chat-client.
Contrast to the below example, client can tell server to
establish connection with client assigned port. That's means client at
initial connection inform server which client IP & port should server
connect to, then close the connection. Then client listen to the
pre-determine port, waiting for server to establish connection. Configure
internet router settings: static internal IP for computer; port forwarding
internal IP & ports used by server / client app. Configure computer
& router firewall.
Revision ‘0’:
c# multithreaded chat-server Console App, both portal server
& worker servers use TCP: TCP_Chat_Server_Console.cs.txt
c# chat-client Windows Forms App,
only TCP: TCP_Chat_Client.zip
Revision ‘1’:
1.
Server & client applications use
XML to send string & parse received string.
2.
Add timeout check in client program.
If server does not respond to client for extended period, client send
‘<CLIENT><command>IDLE TIMEOUT</command></CLIENT>’
message to server and close the connection.
c# multithreaded chat-server Console App, both portal server
& worker servers use TCP: TCP_Chat_Server_Console_rev1.cs.txt
c# chat-client Windows Forms App, only TCP: TCP_Chat_Client_rev1.zip
Revision ‘2’:
1.
‘Portal Server’ listens using UDP transport layer
protocol; ‘worker servers’ listen using TCP transport layer
protocol.
‘Portal Server’ uses User Datagram
Protocol (UDP) transport layer protocol, it provides a connectionless
datagram service that prioritizes time over reliability. TCP transport
layer protocol features three-way handshake, retransmission, and
error-detection, adds reliability but lengthens latency (Wikipedia).
c# multithreaded chat-server Console App, UDP portal server
& TCP worker servers: Chat_Server_Console_UDP-TCP.cs.txt
c# chat-client Windows Forms App,
first UDP then TCP: Chat_Client_UDP-TCP.zip
Revision ‘3’:
Nowadays we can turn a general-purpose computer into a network
router by installing programmable soft-router software.
Therefore, when sender transmits plain-data / plaintext via
multiple switches to reach to the intended recipient, these plain-data can
be potentially eavesdropped or being modified.
Hence alternatively we can implement End-to-end
encryption (E2EE) into our server-client communication. This revision
code is an addition of revision ‘2’ code, adding ‘encrypt & decrypt a
string using standard c#
AES library’ function and ‘Diffie–Hellman
key exchange’ function to both server & clients.
When standard c#
AES library encrypts or decrypts plaintext, it needs a 32-digits
alphanumerical characters as the encryption & decryption key. Both
encryption & decryption key are the same. Hence before sending the
encrypted string, both sender & recipient should know the ‘secret-key’
to encrypt or decrypt the string. However, without the help of 3rd-party,
the key sent by the clients to the recipient server will be in plaintext.
Therefore ‘Diffie–Hellman
key exchange’ function is used to safe-guard the ‘secret-key’.
1.
Client randomly generates 32-digits
numerical ‘client private key’ and ‘primitive root’. We use only numerical
digits (10 characters) instead of alphanumerical digits (36 characters)
because C# standard ‘Math.Pow’ command accepts 8 bytes ‘Double’ datatype
inputs and produce ‘Double’ datatype output. And 31 to the power of 36 can
up to 55 digits. 7 to the power of 10 is 282,475,249.
2.
Client generates ‘client public key’
using both ‘client private key’ & ‘primitive root’. Client sends the
‘client public key’ to portal-server. Portal-server send plaintext to
inform client which port number of worker-server to connect to.
3.
Client connects to worker-server
according to the port number it received. Client sends again the 32-digits
‘client public key’ & ‘primitive root’ to the worker-server.
Worker-server checks if the current ‘client public key’ matches with the
previously received one.
4.
Worker-server randomly generate a
32-digits ‘server private key’. Then worker-server generates the ‘server
public key’ using ‘server private key’ & ‘primitive root’.
Worker-server send the ‘server public key’ to the client.
5.
Worker-server generates the
secret-key using ‘client public key’, ‘server private key’ & ‘primitive
root’. Likewise, client separately generates the secret-key using ‘server
public key’, ‘client private key’ & ‘primitive root’. After the
computation, both worker-server & client will get the same value
secret-key. While 3rd-parties supposing not able to derive the
secret-key when he only know the ‘server public key’, ‘client public key’
& ‘primitive root’ from the plaintext transmission data.
6.
Both worker-server & client can
start encrypt & decrypt the transmission data after derived the secret-key.
Apparently, worker server & client communication using
dedicated network port makes encrypting & decrypting transmission data
easier.
In this sample code, recipient will print-out both encrypted
string and decrypted string it received from sender on the screen.
Code is still undergoing modifications
c# multithreaded chat-server Console App, UDP portal server
& TCP worker servers: Chat_Server_Console_UDP-TCP_AES_main.cs.txt server_common.cs.txt
c# chat-client Windows Forms App,
first UDP then TCP: Chat_Client_UDP-TCP_AES.zip
Example 4: Multiple Clients R/W data from/to a multithreaded Server
application with backend database using UDP & TCP connections in C#
Servers commonly have backend database.
In this example code, a multithreaded server maintains multiple
database connections. The multithreaded server should able to accept
multiple incoming client connections concurrently using server’s ephemeral
ports and should able to R/W data from/to backend database concurrently
using DB’s ephemeral ports.
Our server application and DB program are two separate computer
programs. Both programs can be resided in the same computer or in separate
computers. In this example, the server application & DB are in the same
computer (localhost).
R/W data to database take times. Therefore, preferably to have
multiple DB connections and multiple server-clients web-sockets
concurrently to improve the DB efficiency and responsiveness to clients.
In simple SQL query procedure, process need to first establish
connection with DB before the SQL query. The connection string includes
login DB username and password, DB’s IP address, selected DB name, port
number & etc. After finished one SQL query, process will close the
connection with DB. This process will repeatedly login & logout DB for
every SQL query to DB.
To avoid frequently open & close connections to DB, this
example uses the connection
pool idea. A multithreaded server application login & maintains
multiple connections to DB using multiple threads & DB ephemeral ports.
When no query to DB, the DB connections will be put into wait state to
idle. Connection pool idea
can be used in any type of connections that are allowed to be alive for
extended period by the host; we can create multiple concurrent connections
to the same source if permitted and if it would improve the efficiency.
Multithreaded server-client code portion using the previous
Example 3.
In server application-to-DB coding portion, we will have a
shared queue to save all the clients’ DB queries. Established DB connection
threads will execute the SQL queries whenever there are any in the shared
queue. If there aren’t any in the shared queue, shared queue will make DB
connection threads to idle in Wait state. Server application’s TCP
connections to DB are retained in Wait state. When new SQL queries are
added into the shared queue, shared queue will invoke an idle DB connection
thread to R/W data to DB. Server application-to-DB code reside in server
application.
Worker server-client threads and DB connection threads are
separate threads. When worker server-client thread query DB via DB
connection thread, worker server-client need to wait for DB connection
thread to get DB query result & notify him.
In this example, client establish a new connection with server
application for every new query to the server. Server close the connection after
sent the client the query result.
In this example, a server application’s TCP connection to DB
that does not last more than 1 minute is considered an exceptional
connection error. When exceptional connection errors count more than 10
times, reattempt to connect stop.
In this simulated employee clock-in & clock-out
applications, multiple client applications are able to read or write
clock-in or clock-out time concurrently from-or-to database via a single
server application.
Revision ‘0’:
c# multithreaded server application with database connections
pool Console App, both portal server & worker servers use TCP: ServerWithDB_multithreaded.cs.txt
c# client
querying DB Windows Forms App, only TCP:
webDB_Client.zip
MS SQL Server
MDF database backup file: db_Employee_Attendance.bak
Revision ‘1’:
1.
Server & client applications use
XML to send string & parse received string.
2.
Add timeout check in client program.
If server does not respond to client for extended period, client send
‘<CLIENT><command>IDLE TIMEOUT</command></CLIENT>’
message to server and close the connection.
3.
Portal server listens using UDP,
worker servers listen using TCP. Clients firstly connect to portal server
using UDP, then connect to assigned worker server using TCP.
c# Console App - multithreaded server application with database
connections pool, portal server listens using UDP & worker servers
listen using TCP: ServerWithDB_multithreaded_UDP-TCP.cs.txt
c# client
querying DB Windows Forms App, first UDP then TCP: webDB_Client_UDP-TCP.zip
Same MS SQL
Server MDF database backup file: db_Employee_Attendance.bak
References
·
Book
‘Visual C# 2005 How to Program’ by Deitel
·
Wikipedia
(https://en.wikipedia.org)
|