Saturday, August 13, 2022

Bash Command Quick Reference

 Bash Command Quick Reference


ls => list of files in a folder / directory 

$ ls

 

List of files with permission (long format)

$ ls -l 


List of all files in a directory

$ ls -a 


cd => change directory

$ cd <folder>


Go back to home

$ cd 

Or 

$ cd ~


Create or make a directory or folder

$ mkdir Junk


Locate or search a file.

$ locate filename


To update system db

$ sudo updatedb

"Which" command is used to find other commands

$ which cal

(cal represents calendar command)

Use "history" command to display the history of all the commands you used

$ history

To know what the command does (use whatis command)

$ whatis cal


Copy a file use cp command (cp sourcefile newfile)

$ cp ~/.bashrc bashrc

Move a file (this overwrites and deletes the source file)

$ mv sourcefile targetfile.bash

Remove a file (rm command)

Cat command - "cat" command is used to read the file and display in the monitor

$ cat file1 file2

(merge file1 and file 2 content and shows in the screen)

$ cat >> file2

Append text

$ cat > file3 

(new file)

$cat 

Nano => 

$ nano


Copy the list of files and access from root directory and write it to a file called out.txt

$ ls -al / > out.txt


To see the output of a file used below command

$ less out.txt

Sudo => if we need to run a command with elevated privileges / access.

$ sudo updatedb

$ sudo -s

Exit

$ su - cindy

(This command will switch user to cindy and do necessary work)

$ exit

logout


This appends the output to all.txt

cat *.txt >> all.txt

This overwrites all.txt

cat *.txt > all.txt



Kill a process or application

$ killall firefox 

Bash Commands

Youtube videos to learn Bash commands (Basics)

Beginner's Guide to the Bash Terminal





Intermediate Bash command training


Intermediate Bash Commands (grep, sed, awk, tar, less, gzip)


MacBook - How to take screenshot(s) in MacBook

 How to take screenshot(s) in MacBook





Please see the below link for more details

https://www.take-a-screenshot.org/mac.html



--------------------------------------------------------------------


Kill a process or application

$ killall firefox

Thursday, August 4, 2022

Cron - for Scheduling a Job in Apache Airflow

 

CRON format is used to schedule a Job in Apache Airflow.


Schedule="0010**0"


Field            Allowed values

—–                    ————–

minute            0-59

hour                     0-23

day of month     0-31

month          0-12 (or names, see below)

day of week     0-7 (0 or 7 is Sun, or use names)




Please find the following links for more details

Friday, July 23, 2021

Varada's - My first app (Apple mobile app)


This is my first app. I created it to get the app experience.

Technologies used Xcode and Swift for development.


For app support, please contact varadarajaboopathi@gmail.com


Tuesday, December 31, 2019

Data Migration - Good Practice



As part of Data Migration project, I followed following things.



1. Validation Scripts during development. This will play a vital role in validating data in Go Live and Production (post migration).


2. Reconciliation: There are two kind of reconciliation

  • Technical reconciliation – As part of this, we store the record count of each table (i.e. Source table, Transformed table and Target table)
  • Financial reconciliation -  As part of this, we match the financial numbers as per business needs (i.e. accounts receivable - in old and new system, accounts payable - in old and new system and breakdown of transactions)


3. Actual Data Migration can be done in two ways/approach

  • Bring the old data till last month or a specified date and then last month data during migration day. The advantage in this approach is that you will be migrating Delta on final day and business can do some validation with old data prior to migration day. In this approach we can keep system down time as minimal as possible.
  • Bing Bang migration is a risk-based approach, because success of migration is unknown until migration day. Downtime window is large. In case of any error, migration team needs to fix it as soon as possible in order to proceed further on migration. In addition to these resources needs to monitor the migration flow and record count at the time of migration. 


4. Error records needs to store in a separate table with relevant keys (i.e. the field that indicates uniqueness of the record). These records should be fixed and inserted back into new system after discussing with the business.



5. Additional measure needs to be taken while loading data in SQL Server. (i.e. SP_SpaceUsed ‘<tablename>’ to find the record count, Partition)



6. BDD feature in SQL Server – Balanced Data Distributor is used to read from the source table and write in target table simultaneously. This is because, read operation doesn’t take time but write operation takes more time.



7. Partition can play a vital role in improving the performance.



8. Indexes can improve the performance. I suggest to create Partition and then do Index on top of it.


Tuesday, March 26, 2019

Date functions & Date format in SQL Server



We use lot of date functions in day today coding in SQL Server world. So, thought of preparing a quick reference of commonly used date functions.


1. To get the current date use GetDate() function




2. To get UTC date use GetUTCDate() function


3. See below example show how to get Date only from DateTime


Select CONVERT(DATE , Getdate()) AS DateOnly






4. This example show how to get Time only from DateTime

Select CONVERT(TIME , Getdate()) AS TimeOnly






5. Following example show how to format date


            SELECT FORMAT(GETDATE(), 'HH:mm:ss') AS TimeUsingFormat








6. Use DatePart function to get specific value from Date.

                    a. Year only
SELECT DATEPART(yyyy,GETDATE())





                    b. Month only

-- To get Month                           
SELECT DATEPART(mm, GETDATE())
-- To get Month amended with '0'

SELECT FORMAT(GETDATE() , 'MM')





                    c. To display Month Name

-- To get MonthName
SELECT Format(GETDATE() , 'MMMM') AS NameoftheMonth

-- To get MonthName (3 Character only)
SELECT Format(GETDATE() , 'MMM') AS NameoftheMonth







                    d. Day only

-- To get Day
SELECT DATEPART(dd,   GETDATE())
-- To get Day amended with '0'

SELECT FORMAT(DATEPART(dd, GETDATE()) - 15, '00')






                    e. Year and Month using Format function

-- To get Month and Year
SELECT Format(GETDATE() , 'mmyyyy') AS MonthAndYear







                    f.   The below script will scan "myTable" table from StartDate to EndDate (whatever date range I choose), and iterate through this date-range, and if a record does not exist with any of the dates in this date-range, insert a new record with that particular date in the Timestamp field, but empty/NULL data for the rest of the fields.

DECLARE @StartDate AS DATETIME
DECLARE @EndDate AS DATETIME
DECLARE @CurrentDate AS DATETIME
SET @StartDate = '2015-01-01'
SET @EndDate = GETDATE()
SET @CurrentDate = @StartDate
WHILE (@CurrentDate < @EndDate)
BEGIN
    SELECT * FROM myTable WHERE myTable.Timestamp = "@CurrentDate"
    IF @@ROWCOUNT < 1
        print @CurrentDate
        /*insert a new row query here*/
SET @CurrentDate = convert(varchar(30), dateadd(day,1, @CurrentDate), 101); /*increment current date*/

END





FYI - Starting for SQL Server 2012 version, you can use FORMAT function.




-----------------------------------------------------------------------------------------------------------


Please see following link for further details on Date Functions

Date and Time Conversions Using SQL Server
        https://www.mssqltips.com/sqlservertip/1145/date-and-time-conversions-using-sql-server/


Format SQL Server Dates with FORMAT Function
          https://www.mssqltips.com/sqlservertip/2655/format-sql-server-dates-with-format-function/




DatePart Function






---------------------------------------------------------------------------------------------------




Date Format


Select CONVERT(VARCHAR(10), Getdate(), 112) AS fmtUTCDate




Conversion


----------------------------------------------------------------------------------------------------------------------------


DatePart


SELECT DATEPART(yyyy,'2007-10-30 12:15:32.1234567 +05:10')
SELECT DATEPART(mm,'2007-08-09 12:15:32.1234567 +05:10')
SELECT DATEPART(dd,'2007-10-30 12:15:32.1234567 +05:10')




 ----------------------------------------------------------------------------------------------------------------------------